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Presented is the, second in a series of^forna^ive 
evaluation reports vhich summarizes student abilities and performance 
in field te^ls of * Me.and My EnvironmerjA^ a 3--year life science' \* 
curriculum for 13- tOr 1^-year*old ^educafble mentally handicapped (EMBl) 
adolescents. Discussed are the purpose and interpretation of student 
data for ;\udging a curriculum. Described in relation to development i 
of test items for tha first field test year are aspects such as item 
format analysis. Functional abilities of students are considered in I 
relration to intelligence and achievement .tests, teacher rating 'of * 

! stu&emtsiprobJ^m solving, cognitive development, grouping, and 
.prerequisite ^n^wled^ae. Explained are differences in performance seen 
in results o£ regression 'analysis and differences in performance 
among classes . St ude nt perf or ma nee is ana ly fed for the direct ionaiity 
and maj^ reading,, measurement afnd scalj,; temperaturW, and ^ 
environmental subtests* in Onit I; and for the energ;y, food chains and 
welbs, food energy, weight and Itemperature, graphing, and categorizing 
subtests in unit II. Results ate given wbibh- show that performance on 
25 -items in Onit I was not as high as expected,\ that students in one 
fourth of the classes shoved marked gains on posttests, that one^ ' 
tihird of ^students in nine classes wer6 successful on a^ least one" 
: subtest, knd thatv performance on 19 items in^Unit ^rj'as moderate 

-^one-tbixd of students in six classes,*)erf ortoed succ^sf ully on a:t 
- least one 'subtest) . Considerable revision of both unite ^is. foreseen. 
I Also^ findings are given tp* ind4.cate that ihdJtvidual intelligence • 
tests explain little of^^ih^ variance in strident performance, whereas 
problem splving and teacher rating do expliain the variance*. (MC) r 
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WIeIaND my BIViRONMENT « Vj life g'*^ of ME: AND MY EN V< ROlfi M ENT^' S |n 

prpgrdfni\h^vetoped $pe^ for 13- to 16-ycar-oJd order to rnalce this curriculum available to special educators 

' " * ' '* ' • ^ soon as possible; the ft^ld trials ovei4ap »o that pompl^^ 

f i«)d tests of the irriateriais aure acdomptiihed in three years/ 
Jhe fol low rpg table shows the major stages in tt^devek 
opmeni' ar>d evaluation of M| AND MY ENVmONMENT 



scierKes 

reducable mentally handicapped lEMHJUhiWren, Its develop- 
me^t and assessment, jhe actual materials and their us'e in the 
classroom, the approaches to dataf collection^arid the student^ 
outcomes will'all be iuttjects for ttu^y/. Th*ese evaluative 
act iviti^* might best be viewed in the context of the four arid 
one-half year timeline, for th^developrnent^ testing, and final 



and working titles, of correspond ipg ^terim reports yv^idi 

, are'' anticipated:':.:' ■ . " ' 



MAX)R STAGES IN THE DEVELOPMENT 
^ AND FORMATIVE EVALUATION OF ^ 
y' ME A^ilD MY ENVIRONMENT 



0. Development of Experimental Maten^. Units MV, , 
(June-October 1971) ; ^ 

1« Content Analysis of Experimental Materials 
(November I97l%lune 19721 | 

2I First Field Test ^ , T 

(November 197 IMay 1974) • 

3. First Revision of Units I and 1 1; Refinement of Units 
I I I and I V. development of Un its V and VI 

. (June-September 1 972) J • v . 

4. Second Field Test • : ; 
(November 1972-May 1974) 

5. Content Analysis of First Revision Materials 

^ (December 1972-June 1973) ' V : 

6. Second Revision of Uni»i i and U for 
Cofi^crdal Publicatii)»» ; • : . 

, (February 1973-January. 1974) ' * 

7. First Revision of Unit^ IKand IV; Refinement of 
Units V and VI / ; 
(June-August .1973) ' 

8. Second (Revision of Units IN and IV for 
Commercial Publication 

(February 19V 4.January 1975) • ; 

9. First Revision of Unit V \pr Commercial Publication 
(Februlr^ 1975 June 1975) . 
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CUfiRENT ANDANTK^PATED EVALUATION REPORTS 



0. Plans for Formative Evaluation . ^ "-^ ■ 

(Evaluatfon lssOi! BSCS NEWSLETTER 46, Feb/dary 19721 

\J Arrangthg Field Tests:, Characteristics of Sites Jnd Student 
(Interim Evaluation Report l-^-June 1973) 



2. Assessing Student Abilities and Perf or manci: Ycir 1 
-(tntcrim Evaluation Report 2 -December 1973) 

3. R/v^ews, Revisions, and Data Collection Procedures 
(Anticipated publication-Marcti 1974) ' i \ 

4. The Second rVear: ^Assessing Student Abilities and i ^*^ 
Performance (AnttciFWtcdpuWication-May 1974) 

.^r^ssessing Revised Material: Special Studies 
^ (Anticipated publication -August 19741 

6. The Third Year: Assessing Student Success 
(Anticipated publication-October 1974) - ^ 

* • ■ ' ■ ■ ^ ' ' ■■ ' ■ ■'■ ^-. . 

7. -The Formative Evaluation of ME ANDMY ENVIRONMENT: 
A Synthesis of Findinjjs . ; * *f 
(Anticiiittted publication of f inal report -December 1974) ■ 



^ ^ • i - ' ■ : .K 

■ The materials in the ME AND MY ENVIRONMENT 
progrdHT consign of 3 series of Teacher's Manuals with suggested 
teaching strategies for three years of daily science iniVuction. 
A kit of all equipment and supplies not ordinarily available in* 
a special education classroom is an integral part of the pro- 
g^ann and instruction. The materials do nOit include a student 
text; as the program is designed around ^ti/dgnt conducted 
activities supported by a variety of multi;rsensor^ and media 
instructional materials. Some of th^ese, in addition to science 
equipment in the kit^ include slides, cassette fapesjndividual 
student worksheets/ games, posters, wall charts, illustrated 
booklets, and evaluation materials. The program makes use 
of a 35mm slide projector and .an overhead projector; active 
student involvement wi^ a Polaroid Camera ^nd a cassette 
tape recorder is al^so being field tested. 



0\ 



The- serious reader of this report will likely have 
reviewed, or have access to, the Teacher's Manuals to ME 
AND MY ENVIRONMEIVT. Therefore, information on the 
objectives, science ojntent, and skill development of the 
curriculum will not be described t^ere. (Refer to the frpnt 
material in any unit of the Manual^ for this information.^ 
The' current project and its evaluation afe based upon^ 
severaUyears experience in developing and field testing ME 
NOW, 3 life science curriculum for 11- to 13-year-old EMH 

■*^iidren.^ The ME NOW program and th* first year of ME 
AND MY ENVIRONMENT are available com^^fercjally 
from H^bard Scientific^Company, Northbrook, Illinois. 2 

' Several evaluation re)X>rts are available on these programs. 

NOW. LIFE SCIENCES: A SPECIAL EDUCATION 
PROGRAM, Biological Sciences Curriculum Study, 19/2. 
^Hubliard Scientific Company, 2855 Shermer Road, North- 
brook, Illinois/ 60062. • i 




A SUMMARY OF THE SIGNIFICANT 



FINDINGS TO DATE 



Normative Evaluation Report 1 served 
fourpurpos^s: ^ % * ; 

It defmed the function of evalu^ 
t i 9 n rn c urricul um dev el opme nt 
and presented the evaluation de* 
sign vqt the deyelopfnent of ME 

. andmV^enWronment. 

2. it described the criteria and proce- 
dures for selectihgtield test partici* 
pants, ^ . . ^ ■ 

3; It presented data 6n the actual 
composition and characteristics of 
(ites and of ttie students who parti* 
cipated»in the first year's field 
.■ V "i trials/ ' ■ ■ ■ ' % ' :* ^ 

4. It reviewed the placegient of sti^i* 
dents in this sample of special fdus 
cation classes and drew upon teach- 
er descriptions lo portray more 
fully the variety of young people 
and instructional problems fouTKi 

In these classes. : ! - •S 

* ■ ■■■ * . 



In Formative Evaluation Report 2 the 
Mellowing purposes are served: 

1. The collection a^ interpretation 
, (of student data is placed wfthin the 

context of the overall evaluation 
,desiga •■ V. • -v. 

2. ; Procedures for development of test 
^Uems are described and studies of 

* special problem? arc reported. ; V . 

3. The development of instruments to 
' assess functional abilities of stu- 
dents is reported, an(^ results for 
the'field test group are presented. 

4. ^ Student performance^ related to 
^ • instruction in the two units field* ^ 

tilled is reportedf, J ^ 

5. A statistical analysis of, differences ' 
in performance related to a number 
of variablesiis presented. 

■ ■ ■ ■ 

The following represents a summary of , 
the significant findings contained ip ' 
these two reports. ; : ■ 



'kl:0^^'An analysis of the field test fx>pu!a* 
f tion revealed that over one thi[d of the 
stu(2«nts in the special education closes 
studied had greatly outdated intelli- 
; gence test tscores. Another third 5f these 
, , students! based on teacher reports, were 
I placed in these classes for reasons other 
: than evidence of retanlation.^The 
appropriate placement of 42 percent cf 
the test population could be questioned 
when one took into consideration ten- 
tative evidence of errors in ^asure- 
ment of intellectual functioning cou*. 
pled with pi acemem* of students in 
these classes fsr reasons oth^r than* 
^.retardation^At the same time one rx)tes 
these problems, it must be pbijnted out 
that districts, and teachers seemed to : 
have operated with the best of inten- 
>' tjons in placing students in these special ^ 
classes/at least in many cases. Descrrp- 
tions of students supplied by teadhers 
\>f these field test classes reve^led a wide ; 
range of educational and ^havioral 
problems/Placement in :tteji'$pecjal ; 
education classes may - well 7l^ve.repre- - 
sented thet>est available programming 
for tBechildren^^involved, given existent 
funding and legal definitions of these 
classes. However^ the consequences for 
, the children involved are Extensive hot 
only in the stigma of being labeled re- 
tarded but also in the widening educa- 
tional gap created by their segregation 
from ojher children and from the risgu- 
lar curriculum (see Interim Report No. 
1 fcrsupportin^ata). - * 

; Findings in the present report 
added further concern over the metKod j 
of identifying EV<H children. The indi-; 
vidual intelligence test total score 
appeared to explain little cf the vari- 
ance in performance of students on the 
life sciences curriculum materials. This 
finding held for both of the units of in- 
sUuction taught during the first fieldf- 
test year. Almost half of the variance in ; 
student performance on these materials { 
was explained by a' few items assessing 
problem solving (an aspect of cognitive 
developmental level) and teacher rat- 
ings of students' ability to follow direc- 
tions or work with their hands. It thus 
appears warranted to explore other 
ways of grouping, and even other crite- ; 
ria for placing children in special educa- 
tion classes. The relationship of 
problem-solving abilities to perfor- 
mance in other subject areas also 
warrantsexploration. , ^ 



Student Undentanding of Science 

i ; ^ T h is report pro vid«s 4fie re- 

'suits of measures, of understarJiing of> 
selected science cancef)fe included in 
the ME AND MY ENVIRONMENT 
materials For this reason ttie same cau- 
tion ts in order that w£;s made in the first 
report frtg of results on science by the 
National Assessment Project: *The re- ^ 

1 porting of tbc/results for the first time 
in any subject area will not i^rovide ^ 
measure of the progress of learning of 
the population assessed. The first re- 
porting in a given subject provides 
*bcnc|)-maTk data* against which the 
results of later assessments in that sub- 
ject can be contpared/'* The approach 
10 reporting results used herein is simi* 
lar to that used by National Assessment. 
Student responses to individual items 
'5re shown. Although thei^fficacy of the 
curriculuin must be judged by results 
from tests of th^evised materials, the 

> general levels of f Access in ten science 
areas assessed in the first test are sum- 
mari2edhere. 



* National Assessment of Education. 
Progress. /Report L 1969-1970^ 
Science: National Pesu'lp/July 1970, 
pagex. V ; ^ ' r ^ 
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; Subtest 1 ; Directtioiciatity^sk^ Map 
Reading. At least ha!f of the students :; 
had a rudimentacy grasp of direction, i 
but far fewer could deal with the rela* y. 
tionships amor>g landmarks on a map. 
About half of tf^ students. appeared to 
have a knowledge of compass direcliont 
an understanding felt to be essential in ^ 
this area of instructio n. The standard of 

\ 80 percent successful had been iet by 

: the staff as a necessar\{|evel tojudge the ■ 
instruction acceptable; therefore* con* 
siderabte revision of the materials has , 
occurred^ the level of complexity has 

i^en r&ducQd,and demental knowledge 
of left-right is being emphasized. 

Subtest 2« Measurement and Scale. 
Ratfier than assess directly the students' 
ability to measure things, *this subtest 
was intended to discover whether th#y 
had some td^a of appropriate units j^f 
measureand an approximate concept of 
feet as'a unit of measure. Only about a 
^third of the Students coukl successfully 
deal with these ideas afterinstruction. 
Included jn this subtest/ also« were 
. items or^ the concept of scale — that is, 
I a p proximate representatio is : to scale. 
Less than a thifd of the students were 
successful with these ilems. On review* 
ing the njaterials. theStaff eliminated 
the content on scale as having little rele* 
yance for this population* of students. 
Activities jon the us6 d(^the ruler were* 
revised to. begin on a* much more ele- 
mental level: ^re practice in measur-^ 
ing ^nd estimating ^hort distances id 
feet and inches v^'as provided. A^ mini- 
mum standard waS set for the second 
field test: 50 percent of the students 
should be able to use a ruler to measure 
short distances accurately. 

. Subtest 3. Temperature, This subtest, 
dealt with ability to read the tem|>era- 
; ture on a picture of a thermometer. It 
also involved an awareness of variations 
in temperature in theenvironment,and 
the concept thaf dark -colored things ab* 
sprb more heat than light-colored 
things. Performance c/i individual items 
wa^ surprisingly high, rapgmg from 65 
to 75, percent* Understanding of/the 

* ^roMP of items, however /was consider- 
rab*iy io.wer. Review of the materials 
resulted in the' development, of four' 
activities on temperature wj:ere one 
originally existed, and ^he provision of 
many new opportunities fl>( students 
^to measure temperatures. Tflie'concept 
of heat absorption has be^n deleted 

; 4rom the materials^ *r : * 



Svbtest 4. Envlronrnental Compart^f 
sons! This group of items assessed; 
y understandings central to the curricu'*; : 
lum materials/ although some of tHe; 
items required going beyond what was ; ^ 
^ Included jn instructton«Of a total of ten 
it^ms, most reflected a low level of cor** 
rect responses on the protest and' 
marked gains in correct.responses on 
the posttest. From ha!f to three fourths 
of the studems made correct responses' , 
to ir>dlvidual item^^^n the posttest. To 
remedy some w&iknesses in the mate- 
rials revealed by these iterns, the revised ; 
materials oontain a.great many more 
opportunities for categorizing, observ- , 
ing^ and colrnparing things. Changes' 
we^'e made in the sequence of in$truc; 
^tion relating to the concepts of liying- 
nonliving and life needs. ; 

The preceding four* subtests wtere ' 
designed for Unit J of the experimental 
materials. TheTemaining six Were 
designed for Unit IL ^ ■ 

' Subtest 5. Energy. Between 85 and 90 
percent of the sl^udents had some under- 
standing of the concept that energy is 
required to do work. From half to two 
thirds understood that food is the 
body's source of energy. About 40 per- 
cent of the students could predict at 
which given temperature a liquid could • 
do more work. Two other concepts 
47hich were not a part of direc*.t irtstruc- / 
tton were assessed — a realization that 
t hings contain storecj. energy and the 
recognition that f iving things ultimately 
get their energy fromthesun. Less than 
a fifth of thestude«ts understood these 
ideas. In review, the last two concepts>, 
were judged peripheral to the major \ 
thrust of the unit and were riot develr^ 

«oped in the materials. Revisions for the* 
other concepts involved breaking actsvi- 
ties into smaller steps, providjng more 
experience with forms of energy, and 
inserting practical applications of ener- 
gy concepts. A game was developed 
dealing with balanced diets and daily 
food and eriergy requirements. 



b f ^ thd s tude^rs riecoijiniied the term 
■food web a$ thejporrect llbe* for the 
interrelationships among a pictured 
group of living things, but jess than a 
fif<*^^pf tNk students couW correc}ly 
connect a series of pictures to indicate a 
food chain. For this reason, many 
changes £nd additions to the materials 
were made« resulting in activities deal* , 
ing with what various anirUats eat, a revi- , 
sion 'of a food chain game, a picture 
booklet dealing wUh interrelationships 
among plants and animals, and a series 
of eight review activities in a later unit. 

I Subtest 7^ Food Energy. AboM^ Half of 
I the students had so n>e understanding of 
the idea that all living things ultimately . 
depend on plants as, their source of 
food. Less than half of the students 
grasped the conceijft that plants make 
thefr own food from sunlight and non- 
living materials. To develop these con- 
cepts nriore fully, the sequence of activi* 
ties in the materials was changed and 
some of the plant experiments were 
considerably revi$ed. : 

Subtest 8. Weight and Temperature. As 
in Unit 1, a few items Were used in Unit 
H to assess students' undesrstandir^g of 
measurement concepts. About two 
thirds of the students understood how a 
balance wc;k>, and 43 percent of the 
students recognized 70° as an appro- i« 
priate room temperature. However., 
only one third of the students had an 
understanding of both concepts. In re- 
vising the materials^ the staff decided to 
: recomnr^nd an inexpensive corhmercial 
balance and turn the tin^e* spent con- 
structing balances to using them. 
H^nce/ more time fcould be spent read- 
ing and operating the balance to meas- 
ure the weights of different things. 
Knowledge of an* appropriate room 
temperature^was not an item of direct 
instruction; it, was assunled that stu- 
dents would have worked sufficiently 
with thermometers during Unit II to 
have an awareness of the significance of 
different ternpcrature levels. Revisions 
in Unnit f havebeen made to develop the 
awareness^At that point, rather than in 
ynit li: !n the revised materi&is the bal- 
ance is also introduced in Unit I. ' 



1; Siiitm9.Graphidg.V\^ile severaM 
I had beeri constructed to assess a ^ stu- : 
dent's li fid «irstand ing of graphing, : 
defectis in some of the items rec*:M:ed the ? 
usable number to one — assessing the 
student's ability to recognize the^r* i 
rect graphing of three pieces of i%farma- 
' tlon. Sixty percentof the studefliFwere 
able toselectthecorrectgraph. Because! 
graphs are utilized in various aaivities 
* in the materials to^ummarizeand com- 
pare data« it was felt necessary to devote 
'time to thfe concept of graphing. A new 
activity was developed which provides 
practi<fe In graphing and in reading 
graphs'of variouskinds; In addition^ the ' 
activities which use graphs were ex* 
panded to provide more emphasis on 
the process of graptting itself . " 

■ : '.^ 

Subtest 10: Categorizing. This subtest 
, 'consisted of two items that involved not 
^o'nly the proce^i of grouping things but 
an understanding 'of specific subject- 
matter cojicepts. One item assessechs^ 
whether children^recognized seeds as 
living things. The other involved com- 
. ^ parisori oi^various menus in order to 
1 select a balanced meal. About half of 
'Jtfie students wer^ successful on these 
^ two items. In the revised materials an 
activity was created in Unit l which c^ls 
for students to grow plants from seeds 
and tii maintain pt^tsin'the classropm. 
To further developthe concept of a bal- 
anced meal, the "Full" and Healthy" 
gamewascreatedlorUnitll.^ 



Jn wmlmaM-y 
ured by^5 items related to Unit I of ME 1^ 
AND MY ENVIRONMENT was hot as l i 
high as the project staff felt was neces- 
sary to judge the materials effective. 
E n CO urag ingly . however, e vklences of 
learning were fdund in gains from pre* 
\o posttest results op some^of theltems^ 
In about one fourth of the classes 
niarked gains were shown, and in nine 
of the field test Classes one third or / 
nr>ore of the students were highly sue- . 
cessful on at least one of the four sub- 
tests. TSe extensive^vision of Unit 1 
should result iri an increase in thenum-J 
ber of students understendi ng the con- 
cepts considered to be most central. 

Performanceasmeasurddby the 19 
items related to Unit 11 of M ?Kisi D MY 
ENVIRONMENT^ was moderate. Iri six . 
of thc^dasses one third or more of the ' 
students were highly succejfefu I on at , 
least one of the subtests. Considerable 
revision of UnU II materials should re- 
sult in t>oth a higher level of success arxi 
a greater- number of students experienc- 
ingsuccess. * * V 



ERIC 






INTERPRETATION 
STUDENT DATA 



JUDGING A CURRICULUM 

What is 'the v^^e of>^«yen curri- 
^ culum? Can' its effiiacy an^v«(th^e 
' captured on the first trial by student 
responses to' a few dozen multiple 
• choice items (which are also used fo([ 
the first time)? We think not. Curricu* 
^ lum development has progressed be* 
yond the *'onc-shot rocket*V material 
fired at an entir^^globe of a target with 
no in-course corrections in aim arfd no* 
good idea of where it will land ^ if it 
hitt the target at all. * ' ** 

By^the time the ME AND MY 
E N V m b N MENT curriculum Is 
launched commercially^ it will have had 
many "'course corlkections/* The final ' 
revision w^ll barelV : resemble the first - 
mbdeJ/a portion bf whose testing is re- 
ported in these pages. " * * 

The value of ME AND MY ENVh. 
RONMENT will .be judged eventually 
by many things: 

i—the feeling? of teachq($ using the 
materiials «v 

•-the response and Involvement of 
• : children' ' ' , '-h; 
^ -'the understand I ngs students ex^ 

' press in interviews * j ; 
; -the skills students are able to den)- 
: onstrate or learn 

-the judgment of. Ixperts in the 
fields of science and special educa* 

* tion who review the content 
ii -the perf6rmanice of students on ; 
V test it^ms. ■ ^ 



|F<!^m a t iye i ye Sti m piative 

|£ valuation"';--:^ 

1 1^ Curriculum evatuatlon ay he Biolo- 

V : gicat Sciences Curriculum Study :1s 
viewed as an integral patt of \he devel- 
opmentiol process to prod uce new ourri* 
cula ; The data that ^serve this pur$x>$e ' 
must often be gathered in haste and util- i 
I zed before all the results can be ana- 

< ly^dj This is because the raison d'etre 

. of a curriculum study is to /r^e ava/A 
3blerapi<ff/lt% nev/curricula to schools. 
Onoe a product isin'use« it can then.,be 
assessed in a variety of ways and by a 
variety of interey; groups. ♦ ; : i ; 

' ^ ; There are many purposes one could 
winh-^a single evatuatioh to serve. It 
would be. desirable to be'ab!e to judge 
the ultimate worth of a program and 

, documenyhe impad^^that the program 
has on students during'the tingle the pro* 
'gram h being cretated. first used, ancj; 

. refinedtit is understandable that nnany 

^ audiences — fegislators^-rqsearchers, 
reviewers — are i7bpaVienltfi)risuch data. 
It is also not "surprising that*the func- 
tions of formative and summative evalu- , 
atlon are often confused these are 
new concepts, as is much of the theoret- 
ical structure of the evaluation field. 

-However, the primary audience fj^ 
/br/7?at/W evaluation is the developer, 
whose purpose. mtj$t be to produce a ^ 
viable first product. Before summativk^ \ 

t judgm,ents of worth and assessment of 
outcomes can be made« the existence of 
a replicable treatment must Be estab* 
lished. It the curriculum developer 

: who^ is uniiquely responsible for pro- 
viding the data which show that an iden* 
tifiable' program can be successfull'fL 

: installed and operated in .a variety oj; 
settings. 'This.investigation beconftes^ 
formative evaluatioQ when the devel- 
oper not only, provides this evidence of 
a reliable tVeatment, but identifies 
defects and weaknesses and then rriodf- 
fies the prograrti to produce thje most ■ 
viable treatment possible. Such is the' 
charge tOyWhich the present evaluation 
design for the EMH life sciences mate- 
rials is directed/ It is in this context that 
data on student abilities and perfor- 
mance in tine first year of field testing of 
ME AND MY ENVIRONMENT are 
being reported. 



Assessing StuderitLieamiriB^^lj^^^^^^^ 

^ ;! Many kinds of evidence are being 
col I ccted that infbrm ;us of student 
'learning and abilities. Sources 6f infor- 
mation include teacher judgments on ; 
each actjvi.ty. observers' notes from the 
fulltime observation of icience instruc- 
tion in four classrooms, Interviews of a 
random sample of students, and scores' 
from test Items and performance tasks. 
I At this point in the four-year process of 
^^development and evaluation, these data 
: are tentative and bconcfusive. They are ' 
i fragmented by source and are being 
; reported ^as the results of each coippo- 
\ ne$t are processed/ rather than being 
syr^thesized into a total picture. Si^h a 
synthiesis would be premature; 

the bulk of this report describes 
the use fif over eighty paper ^and-pencil 
test items; These items include ques- 
tions exploring the background infor- 
mation and experiences of 13- to 16- 
year-old students, thek range of perfor- 
mance in &rtain general abilities such ^s 
grouping, measuring, etc., and their 
understapding of information provided 
by instruction. It is easy to put too 
much trust in scores that can be statisti- 
cally analyzed. The reader should b^ 
cautioned that the data based on stu- 
dent responseV are quite tentative. 
; These test hems have been used for the 
first time. New item formats have been 
tried. Among the instructional items 
.some areas of the curriculum*are not 
adequately represented. The validity 
and {I p propria tenes's of some of the i 
items ar^e^till being explored. ' >- 
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; rcpon^ng btthU 

have?These data serve a number of pur- 1 
poses in spite of the limitations and cau- 
tions noted. Several iiodings may have 
far-r^ching implications for the vvhole 
field of special education* Sorie of the 
OSes to which the data have been put in- 
*clude:' i 'y'-i ■ ■ . : 
1 • Verification of assumptions regard- 
ing the backo»4und information 
and skilJswWci't/itudentsinthisage ; 
range possess. ..^^^^^ \ " 
2« : Exploration^ of the relationship of 
general abilities and characteristics 
of this population to success with 
this curriculuml. F , ; 

3. Investigation of the ringe of diffi- ' 
cujty and complexity of tasks to 

• which this population can respond. 

4. Analysis of student abilities and • 
perf ormarice class by cl ass to,deter- , 

. - mine if significant differences in 
i 9 achievement exist. 

5. Assessment of student understand- 
ing of selected concepts and 

jsequences'of instruction. , 
Foj the^ast use these data provide a 
"bench irarj:" against which the^esults 
from further revisions^ can be measured. 
All of these interpretations directly in- 
form the developer of the curriculum. 
Judgments , are possible as to whether: 
curriculum activities begin at the right ' 
level and involve tasks in which students 
can be successful. 

Returning to the rocket analogy, 
thesj^ results have morel:learly defined 
the landing site — the target popul2<^ 
tion. Revisions have fc^en initiated to 
correct the thrust and direction lof the 
materials^ The curriculum is aimed at a 
moving target. Obtaining a "fix" on its 
position at this point in time has prp- 
^vided a reference'tpoint for future 
checks on the rate and degree of move- 
ment of students. Other "in-flight" 
course corrections have also occurred. 
Interim Evaluation Rcp>ort 1 provided 
an initial analysis of the field test popur 
lation. Content analysis by specialists 
resulted in moc^fications of the "pay^ 
load." Tbacher feedback and st^ff 
review also contributed to a number of 
specific revisions/ These are docu- 
mented in Interim Evaluation Report 3. 
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THE 

DEVELOPMENT 
OF TEST 
ITEMS 



Iteftis Created for the First Field- 
T^tYear ^ ^ 

■ 

The^two units of ME AND MY 
ENVIRONMENT field tested for the 
first time dunng rhe 1971-72 school 
year were then called Unit 1 and Unit II 
(Part 1 ). The latter has now been rnodiv 
fied 10 appear later In the curriculum 
sequence as Unit 111.^ 

As a, part of field testing the 47 
adtivities tn thfjsfc two Units, a total of 
83 paper-and-pcncil items were created ^ 
and administered to' students. Just as a 
new curriculum must be tested, so must 
assessment items and procedures. This 
article describes four studies wj^ich 
examine problems in using written 
items for evaluation. Succeeding arti- 

^ clcs report results for items retair^ed. 

5* * In considering the items to be 
developed, attention was directed not 
only to "assessing concepts included in 
instruction, but also to evaluating/the 
abilities and knowledge of ^he sttlclents 
at this point in their development^ The 
items were"^ written and tried out, and 
after carefur^tudy of the validity and 
functioning of each item, 18. were 
judged defective^nd eliminated from 
further analysis. Of the remaining 65; 
two thirds (44 items;\ assessed various'^ 
areas of instruction and were group<>d 
into ten subteUs. One third (21 items) 
assessed four dimensions of functional 
ability related to the maturation, cogniv 
tive development, and experience of 
students. * 



Precedents for Grcup Testir^g of- 
EMH Children 

Trying to assess the understanding 
and ;k)>ow1odge pt educable mentally 
ha nd i capped children preseats many 
problems. Firit of all. some of the chil- 
*dren are non-readtfts, Thosfe who can 
reada encounter much difficulty with 
some kinds of reading tasks. Ppr exam* 
pie/ some gf the children have a great 
deal of trouble following directions. 
For another example, some have diffi- 
culty making decisions based on more 
than two variables. Many to most of the 
children have difficulty expressing 
themselves verbally, especially if this 
resF>onse must be written. Nevertheless, 
the need existed to assess individuals 
and groups of student^ fjy thtflr. under- 
standing of the instructional n^aterials 
befng prepared. / 



Earlier, duiing the^fjevelopment 
arjiS^ielc testing of M£ NOW* (the 
BSCS life sciences curriculum for 11 -to 
13-year-oid educable mentally handi- 
capped children) much effort was 
expended in developing paper-and- 
pencil test items which might meaning- 
fully assess student undsrstanding. f\\ 
length it was demonstrated that these, 
children co^ld respond appropriately to 
Vour choices in multiple choice test 
items which contained a rainimi^m 
^ount of written materiah A pictorial 
oc^gr^phic fprrfiat was used ^n most of 
the^e items. The required response was 
to mark a particular picture or portion 
of a picture Or to mark a word or short 
phrase. In that field test each student 

*A vailable cor^rimerciafty from Hubbard ' 
Scientific Company, 2855\ Shermer 
Road ':Northi>rookJffinois 6006?: 




was provided^witK aljookletcontainiog ^^ 
the test itens. To ensure that the stu- y; 
dents were always together on the right p 
stem, a 35 mm slide of the Item was pro* d 
jected. The teacher then re the entire ; 
Item to the students and gave thenfi time 
to resporxJ. Hence, reading problems ; 
were minim'ze^ the students were kept 
inf the right place, and the teacher could 
check to see that students were follow** 
Ing the directions and marking the items 
appropriately. As a result, information* < 
was^obtained froim students in g^oup 
testing 'situations^ that required only ^ 
about 45 minutes per test booklet tot . 
adnriinister. 

Among the things demonstrated ~ * 
was that students could respond niean- 
IngfuHy to this format and did enjoy the 
testing situation* They were ^le to , 
! attend to the questions for the 30-to-50 
minute time period necessary for th^m - 
to respond to about SOltems relating to. ' 
a unit of instruction * 

• The first year oj/field testing for 
ME AND MY ENVIRONMENT^built 

1 upon this method of assessing kudent; 
J performance. A large. number of multj- 
I pie choice, paper-and -pencil tesj items 
; were developed and tried out (during 

* the, 1971-72 school year). These items 

; v^e grouped into booklets of. ques- ^ 

iions whiqh w re Sfd ministered before 
: and after eac?i of the two units of * 

instraction tested. Since a^different age 
I group of children (with respect to those 
in the ME NOW trials) was involved, 
th2y represented in many ways a hew 
i population. Therefore, in the first year 
of field testing an exploration was made 
I to determine the ability of these stu-^- 
; dents to respond to a variety of item 
1 formats and to several cuing procedures 
i .for keeping their pfece during adminis- 
tration of the items, o. 

i *For a full report see the following: 
James V. Robinson and Rjchjrd R, . 
i ; Tolman. A Formative Evaluation of 
\XME^ H^tOW, Unit iADigestidn and Cirr\ 

culation. Boulder.^ BSCS,.September 

1970.97 pp. (ED 043182). 
I Richard R,Jo\mw.A Formative Eva/- 
i uation of ME NOW, Life Sciences for . 
! the Educable ^Mentally Har)dicapped, 

Bciulder: BSCS, December 1972,305 

pp. (ED 071263). * 



Item Format Analysis 

*• * Thifty two questions in booklet 
form were administered to the first 
.fl£ld*-te$t groOp in November 1971 and 
again^ in MarW 1972. Se^ven item for- 
ma t s w^r c i n duded . An analysis of 
re^nses yielded the following results. 
( One responiv (marked directly On 
drawings): Ten items of the type 
indicated by the following example 
wetreused. n ■ ■■H: 



*V«reilT MS A TEST Tutf 




m OF WTW MO 



WifM PiauHt OF T« TtSf H^fi 
Tif irttW IIVEL AHW TW R«S^«ll»E ' 

■ ■ ■ IC> 



Appropriateness of response wJs 
ve*y high. Only one to five percent 
of the students made no tresponse 
to onelor more of these ite«fls;^nly 
one to five percent marked ^Sbcg. 
.than orie response on any item/^ 
except for Jterhs 17 and 29, which 
drew 2f/ percent and 10 i^ercent 
multiple responses respectively. 
These two items were fai; .imore 
^abstract and difficult th^n the 
other items and receivec| the lowest 
^ number of correct iesponses*- " 
. Inappropriate responses dropped < 
.U» two tb three percent for most of *' 
the items on the posttest^ this for- 
mat wa^iudged to be understand- 
able to EMH children in the 13- to: 
1 6*year-old age range. - 
One response (requiring j)6sitional . 
mark on drawings): Four items of V 
the type shown in the example 
: were used. 



»CK IS A PICTUtt OF « SHOPffUS OSf*. 
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.9 WRK M K ON Tf€ SOW iiWDU OF ThC Sim THAT S£ILS IC£ CRLW. 



The appropriateness of response 
was moderately high.- 0m7 six to 
13 percent of fhe students made no 
response to* these items. From two 
to eight percent marked multiple „ 
e responses; These percentages 
remained at similar- levels on the' 
posttejit! The items assessed ability 
to comprehend and follow direc- 
tions in order to place the mark in : 
the appropriate position for a cor-, 
. rect response. Psychomotor prob- ' 
lems related to the ability to posi- 
tion a response accurately did not 
appear to be a factor in this task. 
» Whi I e th e f ormat itself did not 
> appear to present problems; these " 
items may have tended. to be more: 
difficult than a listing of options 
from which to choose. The format 
Was judged to be understandable to i 
the target population, V 
3. One response {one word, phrase, orjt_ 
^ numeral optipn): Students showed 
?a high degre6 of appropriate 
I ^response to the eleven. items using 
» . \ this format.One example follows: ■ 

■ ♦ 8£LW IS * fWTIWl OF AN tl^lHO MP. ^ • 
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MOW FAR 1$ lT^.BnwF£« CALMIL (WO MISE> : . 
: fWR< M X OK tOiJR CHOICE. ' . r V N^^ 

*ln most cases no more than one to 
.\tNree perce{)t of the students were 
h on respondents pr ^marked more' 
than one answer. This format was 
judged to be understandable to 13- 
to 16-yea^old EMH children. ■ ■ 
One response (story problems with/ 
short responses): The* six itenris 
of this type have appeared to be 
a m o n g t h e m o re d i scri mi na t i ng 
(biseriai correlations with total test 
were bigher than for any other 
items). Yet sludehts encountered 
little difficulty jn responding 
appropriately (see example}: 

?06ER PLWTCD SEVERAL CORft StEK IK VQ POTS Of WlW SOlU 
l©*4TtKS WTH POTS *JT« THE V WER «CW W.: 

Ht PUT fl« POT NEAR THE KlMDOK WD THE OTHER POT IH A DARk\. *:■; 
, anSET. BOTH POTS HAS PlInTY OF FKSH Alt, A20()Ny THErt,; In'^ 

THE PLwas «CAR m groi vew t*a. but THt j ., 
?'.Afas IN THE cLosn ALL um *£iLo»« AJiD DYiw. ' : ^ 

ffARK AR X ON TME TH1H& THAT UHtS NOT TH£ SAK FOR BITil POTS 



'^j soli I I WOTFR I p 'LIGHT | : 



A^ultiple responses or no response ! 
occurred in two to nine percent of i 
the cases on these items^ A similar^:^ 
I e ve I of i r^ppropriate responses ^ 
occurred on the posttest. A^attern 
of! Increased multiple response to ' 
items which immediately followed 
others reguinng a multiple re- 
sponse accounts for most of the ii 
inappropriate res porjses^ This Jor-j 
- mat was judged roffeptable for.use: : 
with 13- to 16-year-old EMH chil- 
dren. ' ' I 
Multiple responses^ (one word and 
picture options): On the four items 
. exemplified by the reprinted item 
below, the percentage of students;* 
who marked a single response was 
respectively one, four, eight, and - 
13 percent. On the posttest this 
l^vel dropped to one percent on 
threeltems and five percent on the 
^ fourth item. From one to three per- , 
■-cent of the students made no ' 
, response to these items. Thus this 
^ format seems appropriate to use 
j with this population. 

: tWRX A.N X A EACH THIKG THAT IS A Pkh Or' YOUR 11(^1 P?)KKT. 





1 SflOKE' 1 


1 water]; 


1 GARSAGE j 


; ("fLflWEis] 




1 PEOPLE 1 


.[^aouDs ] 


1 MR 1 


H m 1 














1 STARS 1^ 


f trees 1 


1 • ^!«.. J 


1 -t^ 1 



[^SWELLS I 



Before the appearance of the first, 
multipfe respofise item, no more 
thantvvo to three children marked 
more thian one response per item, i 
: On the items immediately follow- 
ing multf pie response items, eight 
to 12 children (seven to ten per- 
c e ci t ) m a r k e d m u I t i p l e 
responses - and the same five 
V children accounted for most the 
perseveration of [this multiple 
response set. Only two other items 
in the 32.-item booklet received 
multiple markings^ While response 
^et may have contributed, it seems ■ 
likely that the difficulty and 
unusual format cf these two items 
; were the major cause of multiple 
responses, it therefore was ibn* 
eluded that EMH children in tbe 
;l3-to 16-vear-ol(d age range could 
handle a combination of formats 
including multiple response. j 
S (Continued) . 



I lUm Format'Aji^ysU (Coniinued) ; 
6. .. Multiple response lopen-ended' 
queiti3n$ requirirvg written re^ 
sp6ns€$V: Tor each of * these three. 
: itfcm^^thrce to four words or short 
phraser were required to be written 
in. as the example indicates. 



*The" level of no response was four 
to eight percent, which was consid- 
ered -to be quite low. This dropped 
to one to two percent on the post* 

* test. From fbur to six "percent of 
the responses were incorrect or 
undecipherable. Two or more 
answer^ to 'th^se questions 'were 
produced by 57 to 78 percent of , 
the^students. Performance on the*^ 

■"items was much higher than 
expected , and increased to 67 to 89 
percent after instruction. However/ 
this format does appear to penalize 
some students and require carefut 
monitoring' by the teacher to assure 
that responses are actually record- 
ed for those students who cannot 
get their answers into written form. 
In light of these considerations, a 
written multiple response is a less 
appropriate format for this popula- 
tion than the other formats tested. 
Perfdi^manfce task and written 

^respo/nse. Two items required stu- 
deo;/s to sort ten'dr^wings Into two 
orbiips and name the groups or 
S«scribeihem, (S^^ 
These grouping task^ representeJa 
PT^isch diffei ent format and order of 
test item. The items were manipu- 
lative, applicational, ^d repre- 

, sented a more direct approach to 
assessing categorizing ability than a 
^ult4j|88boiceJofmat permitted. 
In bo^hcases, the children did^ol- 
low the directioa,4«,,s^3rate the 
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pictures into two groups, however^ 
from one fourth to one third of ihe 
stodents prodir^ed'ho apparent 
gr6upir>g5 and were ur^abte to pro* 
vide^labels or descriptions. In a few 
C2ises when labels were present, 
they referred to the top picture in 
each pile, This^result does* not 
Imply an inappropriate /orn^tany 
more than tt reflects the difficulty 
ofJhG task. On the posttest, after 
some experiences In classifying 



things, the incidence of no appar* 
(mt groups and no lat>els' dropped 
to 13 to !4 percent, suggesting th2(t 
use of performance tasks with brief 
written responses, while difficult^: 
IS not inappropriate for this popu- 
lation of studehts. Care should be 
taken to choose tasks which have 
application to activities students* 
might be called upon to perform. 



NAME. 



hflnE YOUR GROUPS 



QWr GROUP 



Task A: 



3 



Five Pictures of^ 
Anjmals 



Task B; 



s/ve Pictures of 
Pollution Scenes 



Task A: 

Five Pictures of 
Plants 



r 



Task B: 

r ■ * ■ 
Five Pictures of 
Nonpollution Scenes 



(NAME) 

THESE PICTURES ARE ALL IN 
ONE GROUP BECAUSE. 



AND., 



,4 



mm.) • 

THESE PICTURES ARE/^LL IN 
■ ONE GROUP BFCAUSF c • . 



/ r 



Stt^dent Understanding of Items 

Things are not always what they 
seem — arxJ test items are ao exception 
to th^s (truism. Students may interpret 
items to totally unexpected ways, or 
not know the^ meanings of key words. 
Hence the items may not be an accurate 
indication of .student ability or under* 
standing at all. To the degree that this 
may occur, the results eithec become 
nonsense or must be carefully qualified 
when interpreted. Thus it is critical to 

■ inves^gate the vaUcfffv^^the meaningful- 
ness/of items. 

In the field lests, item validity was 
explored several ways: by interviewing 
students, by examining \response pat* 
terns; and by reviewing the item con- 
tent in fight of instruction and program 

> objectrves. Hindsight is sometimes bet- 
ter than foresight, and in revlr./v, a tew 
items w^re judged to involve inforcAa- 
tion that was trivial or peripheral to the 

, main thrust of ithe curriculufn.- These 
items were dropped from the analysis. 
Several other items proved to contain 
complex elements v\^ich^ interfered 
with the assessment of the intended 
.concept, Fpr example, ^^/t the following 

, item the students simply could not deal 
with the graph. 
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mt* IM tfHf^ THE TUrOWTWI OF T* suw* im* 

, ' □ □ 0 ■ ■ 

In a few cases, an item appeared 
str aightforward and dealt with concepts 
that required instruction; puzztingty, 
however, most students could answer 
the item before instruction, on the pre- 
test. An example follows. / 



ERIC 
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jfjm n I TMsn tq rw «tr o tiu *n\»im n. 




tau ta Tw Tsf MB KTw 9 m niuitt 



Some unknown defect m the item 
resulted in the erroneous indicetiop 
that most students understobd a rather, 
difficult conpept prior to instruction. ' 

Some items contained technical de- 
fects in art or wordingjwhich cai^ 
their elimination. The two versions of 
the fonowir)g item are examples.^ 
11. ^ . ^ \ 

VftK M X OM m BOY NHO 1& PftOBABLY UXHST. 
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The first version used the term "cool- 
est," which has several connotations. 
The boys are also doing different things, 
which ted to resjson^s for reasons other 
than the understanding intended to be 
assessed. Tho second version made the 
boys mirror im3^s»of each other, and 
the question was changed as shown. Of 
the 47 students intervlevved regarding 
this second version,^almost 75 percent* 
marked the correct response. However, 
40 percent of those who answered cor- 
rectly did not have any jdea of th^9tx>n- 
cep^that white reflects h^iat, nor any 
memory of a scienca activity related to 
this. Thus, the item stiirprovided an 
overestimate of the number of students 
understanding the concept. A more 
accurate estimate would be that 47 
percent of the^roup^sampled grasped 
the idea. ] /' 



Student intarviews have provided 
by far the most useful verification of 
items. About one ^urth of trie ixenii 
reported herein were investigate in th's 
way. For example, after the posttesting 
for each unit of instruction^ over half 
the students in the four Color^b test 
classes were 'randomly selected for 
interviewing, AUogethef, over 80 per- 
cent pf these students were interviewed 
about some of the items. 

The outcomes of item, interviews 
«re discussed in later aHicles, vA,'ere ' 
results of the item are presented. In 
some cases, substantial validation was 
obtained, as in the use of the "conserva- 
tion" item reported under Cognitive 
Development <n the article entitled 
"Functi<)nav Abilities." In other cases, 
items required qualifica^on of results in 
light of th^egree of guessing or misun* 
derstanding that was evident. In several 
cases, the Interviews revealed factors 
influencing studer)t learning ^which 
would have gone unrecognized. (Exam- 
pies are reported for Item 25 in Subtest 
1 and Item 16 inSGbtest2inthearticle \ 
"Assessing Student Performance,"! As 
a result/ the curriculum has been 
revised to deal with these issues. ^ 



Cues to the Right Question * 

. When ability to read and write is in 
question, the use of group-administered 
tests is limited.>towcan one be sure the 

rstudjent is on the right page and marlcing 
an answer to the question being read? 

In the ME NOW cunriculum, the : 
format uted to resolve these problems 
was 35 tmm slides. A slide was made of 
each item, and each slide m^s presented 
on the screen as the' teactier read the 
item to the students. The students 
matched the item on the screen to the 
same item in their booklets and thus 

* kept their place in Iho booklets. This 
procedure seemed to work quite;fwetl. 

While the procedure seemed effec*' 
tive. It vvas also expensive. A study was^ 
undertaken to determine whether two 
less costly procedures for presentl^ing the 
test items might work as welt as th6 slide 
procedure, for items to be used with ME 
AND MY ENVIRONMENT. 

« (Continued) 
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Cues ta'the Right Question (Continued} 

a. In thp first of these procedufles, the 
items were prepared in test booklets 
with ortly one item per page. Th^' 
booklets were 8% X 11 inches, and 
each page was clearly numbered. ' 
Each item was read to the class, and 
after the students had^ re^ondM 
they were told, 'Turn the^page. You 
should now be at the page that has 

T: the number X at the top/' » 
This procedure was based on the 
assumption that the EMH child, age 
13 to 16, can follow this kind of 
structibn and can read nunierals as - 
large as 9S. 

b. In the second procedure, the items 
were again prepared in test booklets „ 
with only one item per page. The 
booklets were SVa X 11 inches. The 
pages of the booklet were of colored 

♦paper, with five different colors 
arranged in^a ;i^ea ting sequence; 
e.g., white, pink,green, yellow, blue, 
white, pink, etc. Each item was read 
to the class, and after the students 
had "responded they were told, 

: "Turn the page. You should naw be 
at the (color) page." ; 
This procedure was based on the^ 
assumption that the E^.H child, age 
13 to 16 years old, can follow this 
kind of instruction and can recog- 
nize the colors used. 

c. A third proceoure used 35 mm slides 
of each item in conjunction with test . 
booklets having one item per page as 
in a and b preceding. This was essen- 
tially^ the procedure used with the 
ME NOW tests. The paper was white 
and each item was numbered. A 
.numbered item was read to thedass 
while shown on the screen. After the 
class had responded they would be 

• told, "Turn the page. The page that 
' is. showing s/iould look like^what is 
on the screen/' The next slide would 



be disp'jyed simultaneou^y 
Four ^ssrooms were used for the* 
study. The basic design was i^itended to 
be repeated measures using posttest, 
only {no pretest administered). The ' 
-design follows with procedures labeled 
a (number) b (color), c (slides). 

' ' ' -A' ^^;■TABu:■1 p-pp^^^^^ 

fitentied Design for Cuinj Stud/ '^ rj^^ 
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The first ihree unii le^lsof the curricu- 
lum were to have been used for the 

, study. Each classrooTn would have used 
,onrt)f the formats with each test. 
^ As the field test progressed, it 

''^ became evident that only twotinits of 
instruction could be completed in the 
year. Tihe design that was implemented 
was as follows: 

> Design for Cuing S tu<ly as Imp] eoieii ted 



tmu Test i 



CUssrobw 



II 



A classroom observer monitored 
each test 'situation. No class or^indfvid- 
ual difficulties were noted as more pre* 
Valent, or as unique, to a specific 
procedure^ Test instructions for all 
classes were administered by the same 
BSCS staff member, to reduce possible 
teacher influence. Four practice items 
v^re*used to acquaint the class with the 
testing procedure at the beginning of 
each test. . 

But because tvvp, not three, units 
' of instruction were completed, the stu- 
dents were not exposed to all condi- 
tions, and an analysis of results could 
not utilize analysis of variance. Instead, 



a correfned "t" test was done for each 
dass between the means of Test i and'2 
for the dass. This requked that the anal*, 
ysis be'done only for those students 
who took both tests, tt also required 
that both tests be transfon'nedtoa com* 
mon scale. The latter was accomplished 
by Iransf orming the scores to a distribu- 
tion with a mean of 50 and a standard 
deviation of 10. * 

^ Table 3 (below) shows the con* 
verted score results and "t" values. 

Though there is some dependency 
on lhe **t" tests (a high converted mean \ 
score ?o (yrie group would force sonw 

, lotvering of means in the other groups), 
there is a consistent pattern for the 
means of the slide procedure to be 
tower than the mean/ of the other pro* 
cedureSwThe difference was significant 
at the .OSJevel for Class 1 ahd nearly so« 
forOass4. 

The study is not conclusive. A bal* ^ 
anced research design, as originally i 
intended, could not be used, and the ; 
results, to the extenf'that they «can be 
interpreted, apply only to the uset>f ^ 
mas^d items requiring about an hour to 
ad minister!^ For a smaller number of 
itprns the^effects of slide use on focus* , 
ing attention or arousing interest are 
not knownJ"While further study is need- 
ed, the results do bring into question 
the necessity for using slides in testfngr 
esp<K^lly in view of their added cost. 

What might account for the indi* 
cated nature of the results? One possi* 
ble explanation might be that seeing an 

Item twice on a slide and on the print* 
ed page — was a partial distractor from 

• full attention to the item in either for-? 
mat. In alt cases, however, each jter v 
was re<ad twice and ample time was 
allowed for all to respond. As the 
opportunity arises, the use of slides for 
lest items will be studied further. 
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A *t" value greater than + 2.09 4ro + 2.26 is likely tgjocvcur by chance only one time In twenty and .probably represents 
0 al (sfgnlficant) differences.. " \ • . i. . 



Continued Use of Test Items 

; The use of pre- and poWeSting in s 
formal achievement-testing framework 
has provided considerable information, 
as the next several articles indicate. 
However, it provided a minimum of 
specific information to writers who -Je 
revising the materials* because only a 
'small number of items were tied to 
individual <«ctivities, with most linked 
to broader sequences of instruction. 
The Items were extremely costly %o 
develop and tirrm.-consuming to ana- 
ly/e* and results were incomplete at the 
time, revision took place, la addition, 
the results were not'available in time to^ 
be of hielp to the field-test teadhers and 
thus were of r» real value in planning 
and checking onlnstruction. Therefore, 
havD^g obtained data or baseline infor* 
mation on the students and their func- 
tional abilitiei^in the first year of field 
testing, the staff planned a $oh>e whaP 
different approach in order to obtain 
performance data in the second year 
of fie^d testing* 

Hence, during thespring of 1972, a 
number of additions and revisions were 
rnade to the Teacher's Manual of Unit II 
to incorporate situational task$ and 
minitests which would provide tmme* 
diate evidence of student understarxiing 
that would be usef ul for the teachers in 
plannmg instruction or review. These 
short evatpation sequences were also 
useful to the project staff in analyzing 
the performance of activities arxi 



4 .; re- 



making plans for revision. The pilot test 
of this approach was judged successful, 
and it was substitute^ for the pre-post 
achievement test format in subsequent 
units arxj their revisions.'Thus, begin- 
'ning with the second year of field test- 
i ng, performance data have been poU 
leaed at the time the concepts vv<!re 
tauQ'ht. Instructional assessnients for 
short sequences of d>:tivities have been 
inWporated throughout the entire set 
of materials* These assessments take the 
formy>f activities in themselves, andjn 
many cases they involve practical 
applicMjoni or actual performance 
tasks ^rated by the teacher.* They also 
incfcid^ a tallysheet forcompirmg infor- 
mation ^n each student or for making 
ratings of performance: Instructions to 
the teacher explain how to interpret 
arxi use the materials. The tallysheets 
and m some cases the student work- 
sheets themselves are Returned to the 
BSCS and utilized as the source of data 
on student performance. For tlie four 
units of instruction tested in 1 972-73, a 
total of 131 assessment items was devei* 
oped; 85 of which can be scored Jor ^ 
individual students and the remainder 
analyzed to. make judgments about 
classroom groups of students. In some 
cases, items which were used in the first 
y^ar of testing were incorporated or 
revised as parts of these instructional 
assessments. 



:' :f;:[:] Jbe aSsessmen| activities were ; 
given the title *'Clues To Success" or 
"Reviews Success/' Efforts Were ^ 
made to present them to the teachernoL 
in.the light of grading the students bu( 
a/a means of determining which stu* ; 
dents were still unsuccessful with the 

materials and needed additional help. 
Thus, these assessment activities were : 
designed.to establish and help perpetu- * 
ate a student success syndrome. The ^ 
data collected will continue to provide 
the revision team of writers with specie 
fic performance data related to each 
activity or small cluster of activities. If 
the^assessment activities also prove to 
serve the immediate need of the teachjsr 

^ in planning further instruction, they 
will remain an integral part ojf the curri« 
cufum materials. The results of this use 
d¥ t(»st items are now being collected 
and wi/i be reported in Interim Evalua-. 
tion Report 6. 
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FUNCTIONAL 
ABILITIES 
OF STUDENTS 



Tests of Inteiiigence versus Func- 
tional Ability [ 

: tntetltgence tests by tradition and 
purpose are deliberately constructed to 
measure a relatively stable, global 
aspect of human potewlial. They origin- 
ated out of a ne^d to screen and limit 
those allowed to .continue in school to 
a well-endowed group who could be 
predicted to be successful at academic 
tasks Within the period of time typically 
allotted for instruction in the^* Suidi ^ 
tests do not provide information that is 
•useful in guiding instruction or In guid- 
ing the educBtional placement of cbtl- 
dren. They simply predict that certain 
children are likely to be unsuccessful in 
performing the academic taski tradi- 
tionally demanded by schools. 

The other major insuument for 
assessing 'school children, achievement 
tests, is also designed to accentuate the 
differences among children rather than 
provide a guide for their instruction. It, i 
too, is a screenir>g and sorting device in 
the tradition aimed at the production of ., 
scholars: The children with which such 
. a system canrK>t(iea I a re routed through 
watered -do wf) versFor^s of the same 
academlcaUy oriented curricula or are 
the childrerr who make up the special 
education classes in the schools; ; 

The problem the teacher faces in 
any dassroom, but critically in thespe- 
cidt education class, is to discover what 

* /a ch child's Jevet of ftunctioning at al^ 
given time actually. is. Oiviy then can thgL 
(earnlng!nia|eriafs be mediated toT 
accommodate the level and rate of func- 
tioning of each child. Note4hat func- 
tional ability is assumed to change as a 

* child develops and to be specific to the. 
^ kind of task, rather than a global meas* 
> ure. it does not incJicate potential, but 

present capabilities. Unfortunately, 
actual tests of functional abjiity do not 
exist. A few are currently being devel* 




oped, little is known of the relation- 
ship of various abilities to instructional 
materials V any discipline, tr, behooves 
each curriculum developer to explore 
the abilities judged uniquely requisite 
for success on specific materials and to 
provide the teacher with tools to make 
the success possible, for each STfident* 
The purpose of the BSCS in sjt^^dy 
ing science for children in special 
tion classes is to pioneer a curricul 
for doers rather thar) scholars. Read 
writing, and arithmetic are not thecen 
tral featufes of this program. Ins 
students do activities and experJbnce 
situations to gain a pracfiQatji^der- 
standing of the world around t hern. The 
materials are intended to bo both f unc- 
t tonal and intellectually stimulating, 
but in the special education setting this 
means calling on a dif fd^ set of skiils* ' * 
AVhat are these skills for the doers, that 
will enable them to cope with' practical 

' problems and situations in and but of 
school?* What abilities influence the 
acquisition of competence tn various 
tasks? And at what levels of functioning ^ 
are the children to be found in their 
special classes? These are concerns of 
the projeet staff^n the developn^m of 
ME AND MY ENVIRONMENT. An 
aspect of the evaluation has been the * 
attempt to identify some of the func- 
tional abilities and to explore whether 
they are indeed influential in the 
success of children using the ME AND 
MY ENVIRONMENT materials. 

Even as the staff set about trymg to ^ 
develop some measures oif functional 
abilities, the schools involved in the 
field tests were req uested to furnish cur« 
rent individual intelligence test data.^ 
The point was to determine, whether 

^ this instrument so widely used in oiur 

: Culture and required by law for place- 
ment in special classes provides educa- 
tionally relevant data for this popula- 
tion. The next article deals with such an 

^ analysis. ■ i:;:!;" 



' Teach<5fs Were asked to rate stu- 
dents on a number of aiteria which 
were judged to contribute to successful 
performance of the science activities. 
These included: / • 

-verbal participation by the stu- 
t ; .(ieptS;";. 

' -their ability to follow direction? 

-their ability to work in. a group 
f -their ability to attend to a task for 
a sustained period 

-their ability to work with their 
ludnds .. . 
-their general attitude toward 
■ school- 

-awareness of things going on 

around them 
-the reason for placement in a spe-: 

cial class. 

More is presented about these ratings* 
and how they ^ff ect success in the next 
article. See the inset for a desct iption of ' 
the rating form and each set of cate- " 
gortes as responded to by the teachers. ; 

: To assess^^sone functional abilities: 
nrK>re directly, a number of tasks were 
developed to>vhich students 
responded. The tds|;^s ranged f rom direc- 
tions to measure something or group 
things to marking multiple choice 
items. The tasl^or items were divkjed 
into four subtests of functional ability: 
Problem Solvrt'ng, Cognitive^Develop- 
ment. Grouping, and Prerequisite 

. Knowledge. While the items had some 
relationship to the science materials, it 
was assumed that they represerhed 
experiential and developmental dimen- 
sions which were unlikely to be affected 
by instruction of relatively short dura- 
tion . The f oliowirig sections contain 
descriptions of these items and results 

s of use. Following these descriptions and 
data, the next article presents informa- 
tion on relationships of these abilities to 
performance. 



Problem Solving 

: i Six items assessed critical aspects 
of problem solving in the context pf ^ 
conducting an e xperiment; all are repro- 
duced in this section. Two of the items 
dealt with knowing the experimental 
conditions (Items 14 and 20). two dealt 
with knowing the question to be 
answered in an experiment (Items 15 
^nd 30), and two dealt with recognizing 
conditions which might influence the 
results of an experiment (Items 29 and" 
31 j . That the items are dearly relate^.in • 
assessing a common skill labeled **prob* 
I em solving" is indicated by the fact 
that the R-b'iserial correlations of indi* 
vidual items with the total score on this 
subtest were above .90 fqr four of the^ 
items, .81 for a fifth item, and .63 for ^ 
the sixth item (NOTE: maximum values 
for an R-biserial are + 125). Levels of 
correct response to thfe items were 
above two thirds of the total group for 
all but two items. On one of these two 
items, 58 percent of the lotal group cor- 
Tectiy answered one of the It jems dealing 
with krjuMnq the experimental condi- 
tions. The iowest percentage of correct 
response (37 percent)^ was iimade on o/ie 
of the items dealing with recognizing 
conditions which might influence the 
restitts of an experiment. Since the stu-. 
dents did experiments, these items cer- 
tainly vvere. related to instruction, in 



several cas^s/cfassroom groups shovved 
large gains an most o^ the items, as the 
chart of pre- and post test scores^ by 
dass; indicates. Nevertheless, \X was felt 
that these items tapped abilities naore 
related to developmental and experi- 
menfal dimensions thanno achieve- 
ment. As the multiple r-egret^sion 
analysisvin the ^ext 3n\c\e shows» the 
problem-solving subtest' explained 
almost 40 percent of the variance in per^ 
f ormancie on th e 44 items used to assess 
instruction. 
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. . . TABLE 5. 
obi pin-Solving Ability! Percentage Choosipg Correct Response for 
Each Item by Class aod Total Group* 





00 


0$ • 


06 


07 


08 


09 


11 ■ 


12 


13 


14 'i^- - 


All 
Classes 


Questions 


» M 

ft- o 
O. A. 

• 14 


t: s. 

a, o. 


V) 


s s 


t & 


i:i 


♦J .: 

■?••<«.." 

.£ 

. 9 


:, 

s. 

15 


44 . 
O 

' a. 

1! 


. 4.» .■ ■ 

■ 0 

' CL. - 

• 19 


.: ^ 4* 

■■■ V» 

a. OL 
^ 109 


14 ' 
15 
'20 
29 
30 
31 


54 75 
46 61 
38 60 
31 27 
S4 81 
46J5 


90 64 

50 64 
30 67 

20 73 
73 55 


78 89 
44 89 
20 BO 
20 40 
44 78 
67 67 


77 76 
77 65 
29 80 
25 73 
54 76 
62 76 


60 65 

5*r<7 

50 50 
58 60 
30 67 
40 42 


58JB3 
58 50 
58 7i 
58 36 
50 67 
83' SO 


79 
64 
55 
27. 
64 
' 79 


76 
71 
• 40 
: 40 
71 
71 


•93 

; E6 

85 
79 


68 
rV44' 
48 , 
17 
44 
48 


€7 78 
56 71 
38 58 
48 37 
44 71 
62 66 


,r ' ■. 7 




Postte$\ only Subtest Performance:** 
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*Complete data were not obtained from classes 01, 02/03* and 04. For purposes 
of the cuing study described in the previous article* pretests were not admlni's- 
teVed to classes 11. 12, 13, and 14* ♦ \ 

**Percenta9es of studenti are shown for each class in which more than one fourth 
of the group scored iaore*than on« S.O, from £he mean.; \ 
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Conslctg^ingithe same six items as a 
subtest, almost one fourth of ttie stu- 
dents could be judged low in problem- 
solving ability, answering two or fewer 
items correctly. This is one starniard 
deviation below the mean for thegt oup. 
One fifth of the students were^hlgh tn 
this ability, answering all six of the 
items correctly. 

; Because of the relationship be* 
Tw^en problem solving and succes^on 
the instructional items, special atten* 
tlon has been givea to the development 
of problern-sdlving skills in the/revision 
of the .-Tiateridls. 
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TABLE 4 
Problem Solving S\ibt«st^ 
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Cognitive Development * 

TKree items: were developed to 
a»ess tf^e ability to deal with logicdi and ^ 
abstract retatronships in contrast to 
concrete, tangible manipulations of 
objects. Atf three are reproduped in this 
section. »^ * • 

^ These items are related to Piaget's 
^ory of cognitive development. One 
(Item 1)« on conservation of liquid j 
quantity, involved the recognition that 
liquid quantity does not change with 
the size arxi shape ottthe container. A . 
second(ftem 17) invoked the ability to 
serially order objects friHn smallest to 
'largest^ The third (Jtem 33\ dealt with, 
the ability to translate a three^week 
interval of time into a specific calendar 
date given the beginning date ^nd the«; 
calendar on which to make the calcuta- 
tron. Evidence that these three items are 
appropriately grouped to measure the 
concept of cognitive development is the 
high R'biserial correlations between 
e^ch item and the total group score on 

1, jm flUJn/ TtSr TUBE HITN Mill W RXMa n INTO A 
KAKEII. fiCM NE FlUID P€ TtST TUK NitN NATU MAIN 
AS SiONI KLOK, MTOI YOUR &ACHII 90 IMIS. 
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Irhis subtest. (The R-bisenal correlation 
for the conservation item is I'OO, for 
the calendar item .98, and for the seria- 
tion task .54.} Normally, th« use of 
tasks like those i::s^ by Raget requires, 
individual interviews, wherelhe student 
explains'his ideas aher he has per* 
formed the task. Because this appi each 
is prphibitivdy e)^nsive, the BSCS 

reconducted a stxtdy to determine 
whether atead^er-demdnstrated paper* 
arKl -pencil, test for*assess«ng conserva- 
tion of liquid quantity might prdve 
equally valid. The sample for this study 
utilized students* froni the four field-' 
test classes in ColQrado. The day after 
the written test was administered. 47 of 
the 70 children wer« randomly selected 
and individually Jnterviewed as each 

. perforn;)ed the Pfaget task in the tradi* 
tional manner* Orange juice was used, 

* and, after the child performed and 
explaijied the task, he was' asked 
whether the beaker or the test tube con- 
'tained {nore orange juice for*hinrto 
drink. Th^ student's explanation of his 
choice supported his other responses. 

• Results were as follows: One of the 
47 children was inconsistent in the 
interview; on his test item he marked 
'They both have the same amount of. 
water, but he could not conserve 
licfuid'quantfty for orange juice in the 
interview. Three children showed. tran- 
sitional' responses; they had marked an 

' incorrect option on the test item but 
aj}peared to grasp the idea during the 
intervievy. The responses of the remain- 
in9»43 children, 92 percent of those^^ 
sampled, w«re consistent with the way 
tHly had marked the test item. These 
results tugqenthat," for this popufation, 
it is possible to assess at least some 
aspects of cognitive development by 
using written items in a group setting. 
For the total group of students in 
^ (Continued) 
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Cognitilft Development (Continued) * 
the field test;54 pctont marked a 
reSbpnse indicating the ability to <3on- 
ser^ liquid quantity; 55 percent Were 
s.uccessful with the seriatton task. 
However, only 18 percent of the stu- 
dents could successfully; translate the 
interval of time into a calendar date-. 
Considering the three itenr^s asa subtest^ 
IS percent of the studenti were hrgh 
performers, answering more than two 
items correctly, whiJe 1 7 percent were 
low perfor^iers* answering none of the 
[terns correctly. The high and low per- 
formers were distributed widely across 
aM of the test dasses in ttie sample. 
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/ Cognitive DeveS^pnent Subtest 
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TABLE 7 ; r 
Cognitive Development: Percentage Choosing Correct Response for 
' ; Each Item by Class awl Total Group* 





00 ' 


OS 


06 


07 


'1 




11 


12 


13 


14 


AU 
Classes 


Questions 

^■ 


IT* 

t £ 
14 


i £ 

4 


s. o 


£ 8 
10 


♦* 

«l Wl 

4 £ 
9 


A. A. 


—33 

••!■: ■ Wl 

'9 


• ** 

o 
o. 

c15 


O 
a. 


o 
a. 


, 109 


■: ■ M • 
M7 : 
33 > 


7750 
13 27 
1313 


50 27 
20 50 
10 17 


33 67 

70 70 
0 50 


62 76 
65 53 
12 20 


30 42 
n 50 
35 30 


52. 67 
67 82 
8 0 


50 
73 
18 


76 
47 
0 




/ 48 
, 52 
9 


57 54 
54 ' 55 
20 18 


; \: Post test only Subtest PerforiMt^ce;** ; , 


Low ; 
(none 

correct) , 


* 35 


















i*;26 




(more^than 
two Items 
correct) 






33 

.■ / 


'%;2Q 


44 




33 


■'. i ; 
■-■*'"■ /;* 




:>.)■■>::■ 


"■';V>15 



♦Complete data were not obtained from classes 01, 02, 03, and 04. For purposes 
of the cuing s£udy described in the previous article, pretests were not adminis- 
tered to classes 11, 12, 13, and 14. - f / 

♦♦Percentages of students are shown for each class in iWilch more than one fourth 
of the group scored more than one S,D. from the mean. 
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Grouping 

Six items were designed to assess 
the ability to sort and classify objects 
according to their common characteris- 
tics. They ire included in this sectiph. 
Two of the items were sorting tasks 
(Items 35 and 36). The^fther four v/ere 
paper-dndi)encii^ items (Items 17/37/ 
38, and 39). All but one of the items, 
however^ were performance tasks rather 
than strictly selection or recognition; 
Most required the aSility to place' 
objiects in appropriate groups, as well as 
the assignment of reasonable labels to 
such groups. All reasonable groups v^re 
accepted in the scoring procedures. 
From 44 to 61 percent of the students 
were able to answer correctly each of ' 
the items. One of the items'(ltem 17) 
involved recognition Of a group. of 
Qbjects sorted on the basis of a specific 
characteristic. Three other items {Stems 
37f 38« 39) involved sorting a pictured 
collection of objects into different 
groups by changing the characteristic 
used'Xp group them . The remaining two 
items (Items 35 and 36)jnvolved sort- 
- ing a set of ten pictures mto two (jroups 
and labeling, or describing^ the groups. 
Thisiatter pair of tasks was scored both^ 
on appropriately assorted groups and/ 
on assigAn4ent of reasonable labels. The ' 
results indicated that if students were 
able to identify reasonable groups, they 
were also generally ab!e ta supply a 
label or description oj^ those group?. 
Considering: the six items together as a^ 
subtest^ 1 6 percent of the students v^re 
high performers, answering more than 
five of the items correctly; 16 percent 
were low performers, answering fewer- 
than three of the items correctly^ 
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V TABLIE 9 

Grouping Ability^ Percentage Choosing Correct Rejppnse for 
Each Item by Class and Total Group* 



Questions 
« 


V XJO - 

- ■■■■■ 44, . 

Of : , lA 
: O 
ou : a. 

14 


05 

» ■ . 


06 

■ .-. ■• ■ 4^ 

: > . o 

■■>-:;;.; g 


07 ' 

■■.£.::.;§: 

Ou ilk. 

10 


08- 
■ Q* a.- 

■■'.v-v\:;:9-i 


09 

- '■■ 
■■■.lit . ' «A 
■ o 


■^;-m^ 

■ ■ VI 

■■■: O 


0M 


iiis 

:ipg;*IKill 


iSiil 




. Clasiej, 

<to iff 

:■■ 

^;?i!;^lrtl09 


17 

35** 
36**' 

■..■■37: 

: 38 

39 ; " 


77 69 

■I 19 
25 : 60 
56 A7 
56 60 


0 55 

i.^. 36 
20 33 
20 58 
30 : ' 58 


56 11 
44 

40 50 
^0 60 
.^0 60 


54 53 

53' 
59 

29 53 
29 73 

29 v^6 ; 


50/25! 
.58 

33 : : '40^ 
58 1 60 
42 i 50 


33 58'; 
IS 

25 45 
33 ; 36 
25 18 


■'*-;.r64' 
64 
65 

^":/-;::?l>:46 
27 

■-::r-^^;:."-36 


53 

. 41 
5ft. 

20 

i 20 
i 40 


••^iStso 

>>■■ ■;'57 
.:aiS69 

^■i?^;^:f?l54 


1^ 


»61 

■35il|?^i'53 
:-t4^*R52 
:3*li47 
:42:SK45 
42#r48 


Posttest only Subtest Performance*** 
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♦Complete data were not obtained frora classes 01, 02, 03, a^^JU^^For purposes of the cuing study described in the 
previous article, pretests were not administered to classes n7T21 13, arfd 14. • \ 

**I^ems 35 and 30 were administered in posttest orfly. \ 

♦♦fperijenta^es of students tre*shown for each class in which more "than one fourth of the group scored more than one 
5,0. from the mean. : ' ; i ' " 



Prerequisite Knowledge 

Six items assessed background 
Information which was not a part of the 
instructional sequence, but which indi* 
cated the kinds of experiences or under- 
^tandin^s that might contribute' to 
Vi4ccess with the materials. All these 
item& appear with thh section;^ .\ • 

The areas of prerequisite know- 
ledge sampled by this subtest included 
two Items ((terns 4 and 16) assessing 
awareness of basic units of linear mea*> 
ure. Thirty one percent of the studenU 
answered correctly the number of f e«t 
a yard; 83 percent answered correctly 
the number of inches in a foot. Another 
Item (Item 38) assessed the ability to 
use a ruler to measure lengths under Vi 
inches within Va inch accuracy. The levcfl 
of performance on this task was unV 
, f ormfy low, with only 35 percent of the 
total group of students succeeding. SViil 
p another item (Item 43) assessed the uite 
of feet as an appropriate, standard unit 
: of measure in estimating distance; 42 
percent of the students were successful. 
' Finally, two Items {Items 13 and I.'2) 
assessed awareness of an appropriate 
^nge of human body temperature. 
Forty six to 49 percemt of the studtmts 
had this knowledge. Considering th/; six 
lterT)s as a subtest of background ir ifor* 
mation, 1 7 percent of the students t/vere 
able' to answer more than four d the 
items correctly/ while 20 pfxcent 
ansvyered fewer than two of tHe Items 
correctly. These items were usecl'as an 
indicator of the breadth of'siudent 
background information ^"elated to 
several forms of medsurement, rather 
than for the specific facts contained in 
the items themselves. Hence, tf ie more 
experience itudents may. have had In 
dealing with length and temperature, 
the more likely (or sq it was ;i jsumed) 
that they woutd be able to answer these 
questions. The students in this, field test 
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were generally more /familiar' with 
inches than feet; but few could accu- 
rately use a ruler. In half of the classes 
students had a very poor concept of the 
size of various standard units of meas-' 
ure end Of which unit was appropi iate 
In a given situation. 

Thr exploration rei^ealed that both ^ 
th€ curriculum developers and the 
teachers in the field test were making 
unwarranted assumptions about the i 
students' ability to accurately use meas* 
uring instruments. Therefore, 'the 
revised activities were deslgni^ to start ^ 
at much. more basic levels in dealing 
with the use of measunng devices, and 
in making measurements and estimates. 

' ' ' TABLE 10 ' V'/'^'^ 

. Prerequisite ^KrKwledge Subtest ' \ 

' Subtest Weighted Scores? 

' ' «e<9htt Ptr Opticffl ^" ^ ^ 
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TABLE 11 

Prerequisite Knowledge: ■ Percentage Choosing Correct Response for 
' ' -E^cli Item by Class and Total Group* , 
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/ Posttest only Subtest Performance*** 
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*Complete data were not obtained from classes 01, 02^ 03, andi 04.. Fpr purposes of the cuing study described In the 
previous article, prttests were not administered to classes HI, 12,yl3, and 14. 

**Itefns 33 and 43!were administered In posttest only, 

♦♦♦Percentages of students are shown for each class In which more/ than one- fourJ:h of the group scored more than one 
S.O. from the mean. / ^ » ' 
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EXPLAINING 
DIFFERENCES IN 
PERFORMAtlCE 




What variables^ account for differ 
ences in.5tudent performance? Accord 
ing to tSie analysis repw'ted in this 
article^ none of the traditionally used 
variables ^uch as age« IQ, or ethnic 
group provides insight into v^y stu-! 
dents were successful or unsuccessful. 
Half of th^ variation in performance Is 

. accounted for by three functional abili* 
ties (problem solvinq; c;rouping, and 
prerequisite khowljedge) and two 
teacher ratings (following directions, 
and ability to work with hands). Such 
results are dramatic becaus3 they 
s u g g e s t t h e p ossibi I i ty of pr ed i ct i ng 
whidf students will be most or least 
successful with the materials. These 
variables^ also focus attention on skills 
/>nd activities wh'ch may lead to 
improved student performance. 

How ' believable are the results? 
Those readers who would like a technic 
cal report should refer^o the foitowing 
section dexribing results of the statisti- 
cal analysis. Essentially, it reports that 
problem s^plvihg, a sfx-item test of f unc- 
tional ability, explained twice as much 
of the variance in student performance 
as did I Q scores, and tbat this result was 
repeated for performance on both units 
of instruction. The* questioning reader 
should also examine the kinds of test 
items from which a performance score 
was derived. The next two ar tides' 
present and discuss the items used to 
assess understanding of the two units of 
ME AMD MY ENVIRONMENT that 
were field-tested; , 

. * What are the implications of these 
findings? The results seem to bear on 
both the appropriateness of the curricu- 
lum being tested and the manner in 
which this population is Wentified and 
grouped. Because many children 
achieved success regardless of a wide 



range of IQ, age/ varying ethnic back* 
ground, a^nd sex, it would appear that 
the material are on target; Itis espe- 
cially noteworthy that IQ, which 
typically correlates highly With success 
in the traditional academic curriculum, 
is not a predictor irt this intentionally 
nonacademic, activity-oriented curricu- 
lum. Since the curriculum also attempts 
to develop further the abilities which do 
explain differences in success, the mate- 
rials wpuid appear to be especially 
ajspropriate. Another Implication of 
these results IS to raise the question 
about the appropriateness of an elabo- 
rate and costly system of individual 
intelligence testing, in light of tKe find- 
ing that intelligence Scores are not 
predictive of performance* with this 
curriculum: Considering the ineffi- 
ciency with which' such a screening 
system is ruq, and the large margin for 
error (as reported in Interim .Evaluation 
Report 1), there is all the, more reason 
to reexamine the entire procedure for 
placement of children with educationri 
problems. Finally, as IntelUgence test- 
ing does not supply diagnostic informa- 
tidn to guide instructional decisions, 
there is more than sufficient justifica- 
tion to attend to measures of functionar 
ability. ; • , 

Were some teachers more effective 
than others in teaching these materials? 
The last section of this article describes' 
the resultsof an analysis of differences 
in performance class, rather than 
among individual students,«;feflecting 
the teaching ability in the separate 
classes. After statistically adjusting stu- 
dent 'scores to compensate, for differ: 
ences in»ability, the staff found two 
teachers (06, 12) to be far more effective 
than others in teaching fone or both 
units of instruction, and tntee teachers 
(08, 09, 13) to be far less effective than 
'others.' A major characteristic v^iich 
might have accounted for these differ- 
ences was the fidelity with vyhich the 
Teacher's Manual was used in imple- 
menting the materials as Intended. ; 
Other than this, the differences proba- 
bly do reflect overall differences in 
teaching ability, i . /Ir 



I Results of Regression Analysis (Continued) 
If problem solving assessed the 

^ame thing as IQ, then forcing IQ to ^ 
enter the regression equation first* 
would resOlt in most variance be^ng 
explained by IQ, wjth little or none of * 
the remair«\)g variance explained by 
problem solving. Table 13 reports the 
results of this second analysis: the tv\^o 
t^ts are not measuring the same thing. 
While ]Q appear^ to account for about 
20 percent of the variance iq perfor- 
mance for both units in th^ second 

'^'^analysis, problem solving explained an ? 
additional^l percent or more of the 
•variance even with IQ entered first. The 
interrelations of I Q, problem solving, 
and perforrpance might be Illustrated 
graphically ias follows: 



; FIGURE 1 , ' 

Schematic Illustration ^ of Variance 

Accounted for by,IQ and Problem 

•^CSoKing . > . ' 



I , Performance 



Problem 
Solving 




Shaded areas Indicate varjance.ln per* 
for^ance accounted for by -the two 
measures. Problem solving accounts for 
most of the variance explained by IQ. 
Hence^ the results of the first regression 
analysis are appropriate to usli^. 

Table 14 shows the interrelation- 
ships of the significant variables. The 
four functional abilities and four 
teacher ra|ings*are only nwderately/ 
related to each other. The .variance in 
student performance explained inde- 
pendently by each of these variables is 
also shown in the first two columns of 
this table. 



TABLE 13 . • i 

Second Analysis:. Variance Accounted for ^'n Regression Analysis. 
' • > with IQ Forced to £ntpf First ; 

(F values significant at' .05) 



Variables ' 








Unit H 


IQ , (Total^ Score) ;Xsi-|ii;;:r-c 
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Results of Regression Analysis 

As an attempt to discover factors 
which mtgtn have influericed the perfor- 
mance of students on these materials, a 

: stepf^AVise multiple regression was under* 
taken. Thivanailyiis was conducted 
both on the 25 instructional items'- 

: related to Unit I of: ME-«A^D MY 
iENVlRONMENT and on thS9 instruc- 

■ 1 ional items related to Urm II. Total 
scores were derii^ed for each sadent 

: baied or. the weights^'tndicated in the 
itol towing artideit ^esd^bing student 

i per ♦ofmaricc data. Omitted from the 
drialysrTwcre aJ^^students for whom 
!cb|Tiplete data was not available ,Some^ 
of these students did not complete^iH^ 
of the two Unit achievement test% or 
background data wa^ available for 

✓ then';. The number included in the anal- 
. ysis was further reduced by thos? stu- 

r dents who dro;>ped out or who entered 
the class durirH} the year, and for yvhcm 

i only partial dMa,was^ava*lAble, Ten 
classes had com^pleted the second unit 

yof instruction; the number of students 
ind'jded in this analysis was 1 09. Based 
on the end^^f-year enroPment of 142 
children in these ckasses, 70 percer^l of 
them were : included in the analysis. 

Twenty variables were entered into 
the regression analysis. These included 
the scorc on the other posttest, the 
experimental class, and , the four sub* 
9icsts assessing functional ^bilities^of stu^ 

: dents (problem solvir^/cognitive devel- 
opment, grouping, and prefequisite 

: knowledge)* One item from the cogni* 

'^tlve dl^velopntebt subtest, which 

[ assesseaability to conserve I'rqiiid quan* 

^ tity, wa$ also entered separately. Other 
variables tnctijded sex of tbe students^ 

; ciironologrcal age, ethnic group, and 

i Wiecl^ler intelligence Scale for Children 

r. (Wl^gJjtQtal 10 score. The ratio of clasS' 
absence to the' total number of days of 

i imtri|Ction was Included tor each stu- 
dent f^r Unit I. ' . . 



^ problem solving, which accounted for 
almost 40 percent of the variance in per- 
formance in tK>th units of instruction. 

In order to avoid misinterpreting 
:he results, a second regre&ion analysis 
was jconductsd Uince the mature of this 
statistic is that when two (measures of 
the same th(1r>g are mdudc^d, all of the 
variance be attribvited to the varia- 



^Finally, eight teacher ratings of stu- 
dents were entered as variables. Each . 
rating scale h&i been carefully defined 
to teachers ^nd included: verbal parti- 
cipation of the stucji^nt in class; ability 
to fptlovvdirections/dbtlity to work in a 
group; ability ^ ati^nd to a task;ability ^ 
td work with orw's hands; general atti- 
tude^ toward school; the student's, 
awareness of subtleties iq what was 
going on around him; and the primary 
reason for the student being placed in a 
special education class^fSee the section 
in the preceding article defining these 
teacher rating scales and the measures 
of functional ability.}^ ^ 

'^able 1 2 summarizes the variables 
accounting for snost of lift var^a^ in 
the first analysis agairnst Unit I and Uhit 
II performance. In th^ regression 
equations the strongest variable was 
alloweo to enter first. This proved to be 



) * . TABLE 12 

nrrst Analysis: Varlance'AcCOuntec! for In Regression Analysis^ 
with the Stronoest Variable Allowed to Enter First 

(F values Significant at -O^for all except Steps 6 and /* of Unit 1) 



bie entering the equation first; the vari- 
ar>ce is not split between the two meas- 
ures). On thie-possibility that total IQ 
could be more closely rdated to prob- 
lem solving and student performance 
than all otber indications suggested, it 
was entered first in the second regres- 
sion analysis. 

(Continued) 

4 • . . 
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Ste£* 


* 'Mi I ' 




Unit: It 


iPrc^lem living 
Following Olrectioni 
Ability to Use Hands 
Absence Ratio {iMH i) 
iSrcHipIn^ AbUtty 
Prerequisite Knowledge 
IQ (Total Score) 
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•»Tcst Class 

Reason for.flacement * 
Chronological Age V. " 
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♦Order ?n whkh the variables entered tfje regression equation. 
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Differmces in^Performance 

Among xClasses 

^^X'' . ' • ■ 

Another v^ay of looking «t differ- 
: enccs in performdr^ce is to use thedass* ' 
room gfoua rwtKdr tfian the indiyidual 
student as tn^ unit of anatysts. Did some 
test classes learn signtm;antly more or 
jess than mhers? Thin is the question 
studied by Covariance analysis. 

To dcc(;^mpHsh this analysis, the 
differences in abtiiti«>s of students in 
* each class at the beginning of the year 
Cor at the start of instruction] must be 
equalized statistically so that the differ- 
ences in class means on posttest perform 
mance do not merely reflect basic 
differences in ability of students^ Such 
an adjustment requires a measure of stu^ 
dent ability independent of the perfor- 
mance*' measure. Total IQ scores are 
traditionally used as the external crite- 
rion. In this instance an investigatioh 
"^was conducted to discover whether IQ 
orproblem'solving &Oility would be the 
iVi se appropriate measure of entering 
ability. This question was raised by the 
finding that the problem -solving meas- 
ure accounted for the greatest, amount^ 
of variance in regreision analysis. It was 
hypothesizecf that problem-solving 
^ ability (or combinations including it) 
would provide the most accurate inde- 
pendent measure of general ability and 
would produce a higher F value, acoen- 
tuating the significance of the differ- 
ences. A second hypothesis was that the 
act;ial variance among classes would be 
narrower if problem sbtving instead of 
id was used. The subsequent investiga- 
tion looked into both measures^ individ- 
ually and together, along with probletV> 
solving paired with ability to follow 
directions. Tables 15 and 16 show the 
results for the foup^ariations of covari- 
ates for each un/of instruction. 

(Continued) 



TABLE 15 « , 
Unit I— Analysis of Covariance 
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TABLE 16 
Unit II--Analys1s of Covariance 

level ' 
**if Sig- 
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DWertnceiHn Performance (Continued) 

R^{iirdle$$ of the covariate, dilf er-; • 
ences dfljjorig classes were statistically 
signitic.^nt. For Unit I; iQ had the 
lowej^t F" value. As the regression anal- 
ysfVsuggested* a combination of IQand 
prioblem-solving ability did not provide 
Qi more significant solution than ifrob- 
Item solving alone. A combination of 
problem solving and ability to follow 
directions (a teacher rating that^added 
to the explained variance in the regfes* 
sion analysis) provided the hlghest*F 
value. In line with these findings, sup* 
p^rt for the second hypothesis was 
seen, in that for problem solving Und^ 
combinations tncludrng it) the variance 
among classes was narrower. This hypo* 
theiis was also suppbc^ed for Uoit fl; / 
however/ the first hypothesis was not. 
«}White all F values obtained for Unit If 
were significant at the .01 level, IQ 
produced an F value larger than any 
other set of CO variates. 

A third consideration in this'study 
of covariates 'nvolved the ddiust<^ 
means for different classes. It was hypo- 
thesized that the teachers and classes 
fatling at the extremes v\/oufd remain 
the. same regardless of the covariates. 
used. This proved to be thfe c^se for^* 
bgjji units. As the combination of prob- 
^Jtern solving and ability to follow direc- 
tions yielded an F value significant at 
the .01 level for both units, it would 
see^ tKat these covariates would be 
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rpost appropriate to use in further anal- 
ysis of this field test group. 

Regardress of the covariate used, 
one teacher, 06, proved to ^be signifi- 
cantly more effective in teaching hoth 
-units of instruction than the other nine 
teachers. A second teacher, 12, was 
significantly more effective than the 
rest in teaching Unit L On th6 negative 
side of the ledger, one teacher, 08, was 
significantly less effective in teaching 
both units. Several other teachers, 
notably 09 and 13, fell .>ell below 
others in teaching Unit ^ 

During the field test year, staff 
meinbers observed the test classes and 
screened the feedback from teachers. 
On the basis of these data the staff had 
identified five teachers who showed 
considerable fidelity to the strategies, 
suggested fn the Teacher's Manual. 
These teachers, 00, 04, 06. 07, and 12, 
also appeared to fully ur>derstand the 
intent of the materials. Four of the five 
classes taught by these teachers yielded 
complete data for analysis. Two of 
these four had the h:ghes^ean score^^ 




and the other tvtro had nrvean scores near « 
the high end of the scale. 

Sevetal other teachers, 09, n,11» 
and 03, had been rated by the sti ff as 
deviating greatly from the intent c ithe 
materiels. Oi these, three had complete . 
data avai|«>le for analysis and two ' 
proved to havp mean scores near the low 
end of the scale; the*other class had a 
lo^er than average mean score. The 
staff had fated the teacher y^tl€s^tas$ 
had the lowest mean score as showing 
moderate fidelity to the Manual. ; 
^ The implication of these firMlings Is , 
obvious. The strategies suggested for 
teaching the activities appear to be an 
important factor in student perfor* 
mance. 
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STUDENT PERFORMANCE 
ON ITEMS ASSESSING 
INSTRUCTION IN UNIT 1 




After study and validation of the 
assessment Uems used during the 
---rwj 97 1*72 school year, 65 items werAi 
r^^kuoed for af\alysisw Twenty one of the - 
retained items assessed the general level 
of functioning of students in the field' 
test. These items v^re discussed in a 
preceding article entitled 'Tunctio'nal 
Abilities of Students.'' l^e renruining 
44 items assessed instruction in the AJ 
activities of Unit I and Part 1 of Unit 
II (the latter now revised as Unit 111).'^' 
. The original assessment items for 
each unif were organized into^a test 
booklet and administered by lijLtest 
teachers Immediately prio^ to^alll^ / 
after, instruction! )n eachsUnit. Testing 
occurred in October )971, F^ruary- 
March 1972, and Mayvlune 1972. Pre- 
tests were not admini^ered to four 
classes participating in a cuing study* 
^^'^^e article entitled 'The (Development 
of Test Items/') Of the ifourteen teit 
teachers involved in administering the 
pretests; only one (03) failed to supply 
the data required for this analysis. 
Three of the fourteen test classes (01, 
03, 04) did not ^complete the second 
unit of instrjctioo. Data is therefore 
reported on thirteen field test classes 
for Unit I and ten classes for^Unit II. | 



Unit i: Analysis of Performance 
Items ^ . " ■ * . 

Twenty five of t|^eV4 items direct- 
ty; related to instruction occurred in 
Unit I. (Theother 19,for Unit II, will be 
discussed In Article 8J Tffe 25 items 
assessed instruction presented in the 26* 
activities of the unit, and the items fell 
into four categories by topic: 1) direc- 
tionality iand map reading; 2) measure- 
ment and scale; 3) temperature; and 4> 
environmental' comparisons. Although 
the results are presented !iy these cor . 
tent topics, the .items were unerpreted 
separately rather than as only ''subtest 
scores/' The statistical analysis 
reported in the preceding articles made 
use of only weighted total scoresf or the 
two unit tests. ■ 

: Pre- and posttest scores for each 
hem, will be shown by subtest, and 
while it is possible to discuss the results' 
of all of these test items in ;terms of 
gains in instruction^ seems more 
appropriate at this point in field testing 
tCL,gfve greater attention to posttest« 
:ra\her than pretest, performance. 
Therefore, unless the percentage of stu- 
dents resporKting correctly^ was essen- 
lially the same on both pre- and' post- 
tests, reference will be made only to the 
posttest percentages. 

The percentage of correct response 
on the earlier ME NOW tesl items was 
characteristically at the 50 percent 
level. In that study many students had 
reasonable and logical explanations for 
choosing options other than the 
intended answers to rrany Items; hence, 
the level of understanding was not 
acoirately reflected in the per6entage 
of students, choosing the "right'* 
answer. A similar condition also has 
occurred in the ME AND MY ENVI* 
RONMEWJ assessment. In fact, 40 
percent ci the Unit I testt items have a 
qualitative scoring key, with some 
responses given credit as partially cor- 
rect. The summary table of responses 
by class, however, reflects onfy the per- 
centage of students choosing the most 
acceptable answer for each item. The 
reasons for choosing other "answers" 
provide a separate iustification for 
Inspecting each item separately. 



interpretation of Results 

The problem of assessing the learn- 
ing of EMH students is not resolved by 
producing am "achievement test/* since 
the question of what standard to apply 
to performance on these items or sub- 

> tests is one that is difficult to answer Jt 
is urvealistic to expect all children to be 
able to answer all items,''when ^ wide 
range of difficu*Jty and a range of topics 
are involved. Answering even one or 
two nK)re items correctly on the post- ^ 
test than on the pretest may represent 
considerable learning for some stu- 
dents. The items represent the staff's 
judgment of key content that should b9 
learneci, naf all learning that^n be.* 
expected to occur in instruction. Some 
areas of learning were not assessed at all; ' 
others w^e explored cnly through 
intervieuvs and are not r^orted here. 
Notably absent are measures of observa- 
tion* skiflfs and problem-solving skills 
th^t are dfirectly^ related toMhe mate- 
rials^ $tudent$' attitudes also are t\ox 
reflected here, atthough measures are 
reported elsewhere. The eilects of thir 

' curriculum on self-confidence,lsocial 
participation', task orientation/ and 
general response to school have not \ 
been assessed! 

. Efforts were made from the begin- 
ning to assure that use of the items in \, 
different classes would be comparable. 
The tests. were administered by the 
teachers* using the same Instructions. 
Some training lyas provilrded at the 
beginning of the school y&atr, and addi- 
tional wrhteninstructionslvere sent for 
each of tN four teeing periods. Even 
with these precautions, however, condi-. 
tk>nsand prooedu*res were not standard- 



ized. (For example, total test time for 
the pretest on Unit I ranged from 30 
: minutes^to 85 minutes and was distri' 
buted over one to three day 
f *The test items themselves were 
undergc^ing their first field test. Individ-'* 
ual interviews with Tiudents were 
conducted to Validate some of the 
items, and « ss a result^ approximately 
20 percent of the original number of 
items were elipiinated as defective. 
Undiscovered vveakne'sses may* remain 
in some of the items retained for^ 
analysis. \^ / ^ ^ v 

Only a small numUer of Stems origi- , 




r>dlly existed for each instructional area 
to be assessed. The 20 percent loss of 
defective items contributed to an 
imbalance in the areas actually assessed. 
Although some broad understandings 
were assessed by interviews, teach< 
ratings* and observdtk>ns« and 
reported elsewhere* the results and 
elusions both there and here shqult 
interpreted with caiitton. 

With all these qualifications Ci 
ered/ the test items do represent the 
best judgment of the staff as to key 
concepts to be learrted in the materials. 
Even though the results should not be 
used as iummative evaluation of stu- 
dent learning, they do provide a clue to 
the degree of learning occurring. Com- 
bined with the informationon the func- 
tional abilities of this population, 
expectations of student response can be 
readjusted* and revisions designed to 
further enhance learning. 

A summary of results from the use 
of the test items follows. 




Unit i; Subtest j; Directionaiity 
'andMap Readinja'-v:/":- 

; V: ; T3We 17 shows a summary of Sub^ 
: test I statistics, indi vidijal items, tfieir 
scoring keys/and the percentage of stu- 
dents in the total group who selected 
each option on thepre-and posttestare 
shown. The percentage of uudent 
class, who chose the "best" res^dnse to 
e»dh item is shown in Tab le 1 8 J 

These items had a' wtde/rany^d of 

^ difficulty, from Item 25, whch almost 
none of the Uudents answ«%^d cor* 
rectly, to Item 2, which about two 
thirds answered correctly. About one 

J .third of the students ia 4 of the 13 

l^asses were able to answer moM of the 
items in this subtest; two thirds or nrore 

^ of the sttidentr in two classes, fewer 

' than two items. ^ V ^ 

. Of the five items relateCS to direc- 

/ tionality and. map reading, two (Items4 
and 6) involved j student's ability to 
orient himself in space. These two items 
required a knowledge of which side of a 
map is north, a klnowledge of compass 
directions, and ocif ntattbn for all direc- 
tions when presejiit'ed on a map. On the ; 
posttest, rcju^fily two thirds of the total 
group placed north at ^^*4pp of the 
map. When given north, hal f of the total ; 
group corr^itlyJabeled the other three 
compass directions. 
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l : T^ Items in Subtest f dealt with 
: ; locating things on a map. One (Item 25) * 
^ asked students to trace a street route 
from one landmark to another. Fiftyj 
percent of the students did so on the 
posttest. Hovvever^ for another item 
(Item 6) only one third of the students 
correctly Indicated what direction one 
landmark was from another i-r the same ' 
percentage as on the pretest. The most 

* difficult item in this subtest dealt with 
the ability to locate the intersection of 
two streets on a map. This item required 
that the student be able to recognize 
that a signpost on a i^treet corner pro- 
vides the information of wliat two^ 
oppositely oriented streets to find and 
trace to their intersection on the map. 
Only six students out of 172 (or three 
percent) correc.tly did this on the post- 
test, and all but one of these six stu- 
dents were in the same classroom. 

-A random sample of students was 
. interviewed about Items 2, 4, 25, and . 
26 in this subtest. For Item 2, typical 

. responses to the interview were to point 
to the top of the map to Indicate northJ 
For Item 4, identifyjnigeast, vvest,and 
south on the map was more difficult; 

. however, the intent of this question, 
too, was clear to the students. For Item 
25, locating an intersection on a map 
involved numerous problems. Many stu- 
dents were not able to derive the names . 
of the streets from the street sign in the 
picture of Bill's house. Some simply 
placed a mark somevvhere ori the map 
directly below the street borner in the 
picture. Some tried to track the direc- , 

* tion of the street in the picture directly ' 
down onto the map. These related but : 
I n CO rrect efforts probably accounted 
for many of the responses coded 4 and 5 

* on this item. The Inability of many stu-* 

; dents to find the two different streets : 
around the margins of the map and 
trace them to their point of intersec- 
tion was a major problem identified by 
student interviews. Even when given the 
: streets and their locations on the map, a 
. number of students were unable to fol- 
low the two coordinates to the point at 
' : (Continued) 
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DirectionafitY and Map Reading (Continued) 
which they crossed. Haying nnarked a 

'■ location for Bill's house, nuny students 
understood, in (tern 26« how to start 
from that point and draw a line showing 
where he went nn his bicycle. Some stu- 
dents were confused by the art work 
and labeling of the streets on the map. 
Still other students simply drew a line 
indicating Bill going any place on his 

;|bicycle and did not understand that 
they were to find a path from home to 

* school. • * ■ ^ 

^ Table 18 Indicates the percentage 
cf students in each class whose total 
$ubt^t^sco^e was nnore than one stan- 
dard deviation above or below the total 
group mean. This is essentially a table 
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showing the percentage of students in 
each class who were high performers^ 
answering more than' three items 
correctly, or low performers, answering 
fewer than two items correctly. (Note 
that partial creditwas possible on three 
of the five items.) / 

Considering Items 2^ 4, 6, 25, and 
26 as a'Subtest, 21 percent of the stu- 
dents in the^otal group answered fewer 
than two items, and only '20 pefcenw 
answered more than three items, indi* 
eating a generally very low level of 
performance* in directionality and map 
reading after instrucJ ion (mean: 1 1 , 
S.O.: 7, maximum score possible: 25); 
Based on these results, additional activi- 
ties were v^itt«n into the materials to 
provide practice in recognizing and 
using compass directions and in making 
use of a map to trace routes from one 
landmaric to another. Investigation of 
Item 25 on locating an ex3Ct"point at an 
intersection suggested that students had 
difficulty using coordinates in a/?^ way; 
therefore, an actiirtty was written dead- 
ing^with fraphing, and practice was 
provided in locating a numoer of coor- 
dinates, with the teacher*s\attention 
drawn to using two variables aHd locat- 
ing intersections. 
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. TABLE 18 — ' / > 

Subtest I {Directionality t ftap Reading): Percent. Choosing the Correct Response, by Class 
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♦Percentage of students per class whose total subtest score was more than one Standard Deviation above X)r below the 



Unit I, Subtest 2; Mmasurement 
andScale 

Table 19 shows a summary of Sub- 
test 2 statistics. Individual Uen\i and 
their scoring keys are aisp sbown^ along 
with thp percentage of students in the 
total group who selected each option on . 
the pre- and posttest. Table 20shoV$ 
the percentage of students, by dd^s. 
who ch/ose the best response to ^ach 
Item. The general level of performance 
on.these Items ranged frdm one fifth to 
one half of the students who answered 
an item correctly; 

The six item^ in this subtest assess- 
ed ability to estimate distances and an 
understanding of the use of scale to 
determine distance on a map. Two of 
the Items (Items 7 and 41) dealt with 
actually estimating distance^ which 
required students to have at least a 
rough approximation of the meaning of 
units of length; sutfi as feet. Less than 
20 percent of the totaLgroup made 
reasonable estimates after Instruction, 
although alhtost twice this many (39 
percent) indicated that pacing a dis- 
tance was a proicedure that could be 
used to make such estimates. Three of 
the items (items 12, 16, and 23) dealt 
.with using a 5Cdte given on a map or 
drawing to determine length or dis- 
tance. Over one third of the total stu- 
dent group proved successful on these 
itemsonW posttest. 

A ranqbo] sample of students was 
interviewed regarding their understand- 
ing of Items 7, 12, and 16. On JterA 7, 
almost two thirds of the students inter- 
viewed could correctly explain that a 
pace was a step. Only one in ten stu- 
dents,, however, knew how.long his pace 
was* Loss than half of those vyho mark- 
ed the correct response answered cor** 
rectly what a pace was or how long 
' (Continued) 
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MMSurement and Scale (Continued) 
one was. For Item 12, the intervSe^s 
revealed that many students had no 
notion of how to use the scale provided. 
A number of those who marked the cor> 
rect option said they just guessed or 
t^|,ouQht that the real boat could tie 
thirty feet long. Some answered the 
question only In terms of the pictured 
drawing and appeared to have no con* 
oept of It as a scale drawing. 

* In the interviews, Ibout two thirds' 
of the students could actuelty trace on 
Item 16 the route from Caldwell to 
Borse with a pencil. Of those who had 



marked the correct response, only ha\% 
understood the use of the scale for 
miles; the' others guessed. A signifk:ant 
fir>ding was that one student in three 
did not even understand the map 
symbo1ism.which indicated where the 
two cities were and the route between 
them. Some traced beyond the two dots 
signifying the dties; others did not 
follow the road. 

Table 20 Indicates the percenti'lge 
of students* in each class whose total 
subtest :score was more than orb stan- 
dard .deviation above or below 0)6 tota^ ^ 
group mean. Hi^ performers on fSiis 
basis obtained a scor^of 1 7 or more;t7Ut 
of 30 points, or more than three o¥ the ' 
six items correct. Low performers 
obtained less than' three points, which 
rneant answering lejs than one complete 
item correctly. (Partial credit was possi- 
ble on two of the six items.) 



Comsiderimg the si x items as a sub- 
test/ 20 percent of the students 
answered more than three of the six 
items correctly, while 18 percent 
answered none correctly. Thus the level 
[of performance on measurement and 
scale was also quite low after instruc* 
tion. The materials have been revised to 
provide^ a great deal more practice In 
estimating and measuring very short disr 
tances. Revised activiti^ begin at a • 
much more basic level on the use of a 
ruler, and studeints are given many more 
opportunities to measure short lengths 
with it. The concept of scale,*and prob- 
lems dealing with scale^lhave been elimi- 
nated entirely from revised materials. 



- ' • TABLE 20 ; 

Subtest 2 (Measurement & Scale): Percent Choosing the Correct Response, by Class 

Field Test Classes ' ' 
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Unit I , Subtest 3, Temperature 

See Table 21 for a summary of Sub- 
test 3 statistics^ the four items, keys, 
and pre^ arxj posttest results. Qass-by- 
class results are shown in Table 22 . 
Considering the general level of perfor- 
mance on-individual items, three 
fourths or more of the students answer- 
ed three of these four items correctly 
before instruction* Except for Item 20 
' little change in performance was noted- 
on the posy est. i 

: One item (Item 3) assessed the stu- 
dents' understanding of how to read a 

■ ■ i'i;^' -■ : [ .;■ ■ ■■: :imi Z\W&fMff- 
Sunmary Infonnatton for Subtest 3 > : 

[ (four,1 tesis) 

This group of Ui^ms iwasured: ^ ' ' 
—understanding of how to rtad a ther- 

V mometer;,(Item 3) "'}■: ''■''■■;:./}''-f^' * : 
**awftreness of where outdoor temperatures 
; are likely to be highest { Item 22) 
* -knowledge tha.t dark colors absorb more ' 
, colors (Iteni ll) 

--recognition of a record of outside ; 
•; temperatures e*f re ssed as a gi^aph . ^ 

■jli'dtew. 20) /^ r:.:'/ 



; ; ; ! ; Subtest Weighted Scqres: :^ 

- ; ^ * . Weights Per Optloi* 

Item :- l- ■ -a-'-: 3" :"4^^^'- S'-^-^-S-'^ 7 

Subtest Statistics: * ; : 



Maximum possible score: ; 20 
Range of Scores; 0-20 
Mean: 11.7 [ 
"Standard Deviation: 7 



thermometer* It was answered correctly 
by three fourths of the students on the 
posttest, indicating no gain from pretest 
responses. A second item (Item 22) 
assessed awareness of where tempera- 
tures are likely to be highest outdoors. 
Again, three fourths of the students . 
resptonded cprrectly on the posttest/ 
and again this represented no gain from 
pretest responses. A third item (Item, 
11) assessed knowledge that dark colors 
absorb more heat than light colors. 
About two thirds of the students 
answered this* item carrectly on the 
posttest^ The fourth hem (Item 20) 
assessed recognition of a record of out* 
^ side temperatures expressed as a graph 
and compared with a graph of indoor 
temperatures. Almost four fifths of the 
students answered this item correctly 
on the posttest. ' \ 

A t^ancfom sample of students was 
interviewed about their understandiiig 
of Items 3, 1 1, and 20. Almost 40 per- 
cent of the students interviewed could 
correctly identify the Fahrenheit scale 

'3. . ^- J 
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given in Item 3, as well as give a reason- 
aWe explanation f cJ- answering the item 
as they did. In one out of five cases, the 
students acknowledged that they 
simply guessed on the item« Of those i 
who marked the cofrect response for 
Item 11/40 pfercent did not have any 
idea of the concept that white reflects 
heat, nor any memory of a science actlir* 
ity related to this idea. Thus, the results 
(Continued) 



2o:" ■ . y ..■ ■. ■ 

1C «MMI 9 fOPCMTWC. OK CMM SMBKI TIC 

iPnntM ostsm 9 « scmounm. m enct mm m 



I" 



Is/ 



IS 



1 9 i « S I 7 



m« Nl t « tic CMW WKM TOU TNI» tNIMl W WTSIft lUVfWTUlC. 



22 . V* 



uoDK w TIC nawn of nxts » tic ri^iinfoiT. tw 

TmUIWC IMS TWO IN TIC ClRCLa MU III EACH f ICTUC. 




^ M X ON TIC naoK. wicn rou think noiu nwe tic 

IflGHEST TUVtMTWC ON TIC VWC SWMY 



it If 



Temperature (Continuwl) J * 
are an overestimate of .Uudent under* 
standing of this concept hem 20 
involved a minimum of guessing on the 

• students' parts. They responded on the 
ba%h that outside temperatures vary 
whtl/e^ inside; temperatures ref^^^in the 
same* Other than noticing this differ- 

J ence In the shapes of the graphs, most 
students did not understand the graphs 
and could not read them. Many could 
not visually trace the points on the 
graph to its' scales for the temperature 

i or^hcday. 

When this group of items was con- 
sidered as a^ iubtest, the average was 
determined as angering two of the 
four items correctly. Twenty two per- 
cent of the students answered all four 
items correctly; 17 percent ansvi^red 

- none correctly (mean; 12,S.O.: 7,max- 
tmum score possible: 20). Table 22 
shows the perccmtage of high^nd low 

•'performers, by class. Compared to per- 



formance on the other subtests, stu- 
dents were slightly more successful with 
this group of Items. Considering pretest 
levels for this subtest, however/gains 
f/om* instruction were disappointing. 
Several changes were made in the 
related activities during revision. Where 
there was one activity involving temper- 
ature in the original materials, there are 
now four activities. One of the new 
activities provides practice in reading a 
thermometer^ while two more provide 
many opportunities for students to use 
the thermometers t6 'measure tempec^-, 
tures. The fourth new activity on tem- 
perature has been developed on reading 
and maldng graphs^ providing much 
practice in this skill. 




Subtest 3 (Temperature): 
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Percent Choosing the Correct Response, by Class 
- field Test Classes ' 
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Unit I, Subtest 4, Environmental 
Comparisons 

; Table 23 contains a summary of 
: Subtest 4 statistics, items, keys, and. 
pre-posttest results. Class-by-class 
results are shown in Table 24. Ten Items 
were included in tht$ subtest/and five 
allowed partial credit for some rc- 
sponses. Many of the ten Items had a 
low level of coi rect response on the pre- 
test and showed marked gains in correct 
response on the posttest. From half to 
thrcte fourths of the students made cor* 
red responses to individual items on the 
posttest. . «■ 

Sutnnary Information* for Subtest -i^i^S^ 
Envirdnmental .Cata>Ari«>ns:^?^:S^^ 

• This group of Uems -aVsesseidtf ^'^^^^^^^^^^^^ 
;r-the awareness of. how things! &ri.^l:>i^-,;:,<i;^ 
! : a f f ec ted o r change J 1 r an cjiperimeh it 

the recogn 1 1 1 oi» of concrete br»Sl:;; f 
intangtbiCi nesr^at-hand or; remote 
h;*features of the envlroninenrt 

an understanding of the cateaories ? • f 
■v living-nonliving (Items 19, 28) ■ : 
^-recognition of nones^entialsiversus 

:necess1t1es for life (Item 2;)fe^ 
-—knowledge of human ^nd plant needs • : 
■^^.■/ for. life (Items 32 

; .■■ . Subtest Weigh ted \Scores;J|lg^^^^^ 

- Weights 9er t^tion}'MM 
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Three items (Items 8, 9, and 10) 
were concerned with awareness of how 
things are affeaed or changed in an 
experiment which the students had per- 
formed, (the Information assessed had 
not been called for by the experiment, 
however.) On the posttest; f rom-one 
half to three fourths, of the students 
marked the appropriite responses. For 
many classes this re^sented a constdv 
erable Increase in correct responses. An 
ifem requiring multiple responses (Item 
1 3) assessed the recognition of concrete 
or intangible, near-at-hand or remote 
features as components of the environ* 
ment. Fifty eight percent of the stu- 
dents idenvif led most or all of the items 
as components of the environment on 
the posttest, a dramatic increase over 
pretest performance. Three items 
(Items 19; 24, and 28) assessed an 
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understanding of the categories llvingA 
nonliving. Two of these required simple 
recognition of the categories, and on 
^ these items, from two thirds to fotir 
' fifths of the students responded cor*, 
rectly. However, only half of the stu- 
dents could supply the label livingA 
nonliving to a set of items which 
required this description. ; 

One item (Item 27) assessed recog- 
nition of nonessentials £is opposed to 
necessities for life. Thirty nine percent 
,,|'of the students responded correctly to 
.this item, three times the pretest per* 
centage of correct responses. Finally; 
two items (Items 32 and 33) required 
written*iresponses listing human and 

(Continued)' 
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Environmental Comparisons (Continvied) 
plant needs for life. From one half to 
two thirds of the students responded 
correctly on the posttest. f 

Table 24,shows a summary of high 
and low performers on this subtest. 
Co^i3§)yng all ten items is a subtest. 17 
peMBfixjtPof the students .in the total . 
group answered eight or more of the 10 
items, while 20 percent answered fewer 
than four of the items (mean: 28,S.D.: 
12,.maximum score possible; 50). The 
level of posttest performance on this 
subtest was relatively high, with the 
average determined as answering cnore 
than half of the items correctly. Individ* 
ual I classes demonstrated considerable; 
gains on various items as welL To 
remedy some weaknesses, the revised 
materials vyere given a great many more 
directions and examples for categoriz- 
ing, observing, and comparing things. 
Revisions were made in the sequence of 
instruction related to the concept of 



Hying/nonliving, and changes were 
.made in the treatment of plant and ani- 
mal needs for life. 

In summary, the perforrr»orir:e on 
25 measured items related to Unit 1 of 
M E AND MY ENVIRONMENT was 
low. There were evidences of learning 
on some items, as indicated by gains 
from prevto posttest scores, but the ' 
overall results do not reflect levels of 
learning considered acceptable by the 
project staff. Ari increase in the number ■ 
of students understanding the key con- 
cepts is sought in the extensive revision 

of Unit I.'. * •■ 
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Subtest 4 (Environmental Comparisons): Percent Choosing the Correct Response, by Class 

Field Test Classes 
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In the Unit II experimentalis 
^f r^tv"'^ test, 19 Items related 'to , 

llj.;,; instruction in the 21 activities of Unit|^^l ' 
IjTI^'? 1 1, Part 1 (Unit III in the revised edition) 

lined for analysis. These iterm^; J/! ■ 
grouped Into six categories by|§|$! 
(i^ith topic numbers sequential tf?f 
|S||to subtests in Unit I): 5) energy; 6) food SI 
ylllchains pnd webs; 7) food energy; 8) l^^"^ 
l^weight and temperature^ Ql^aphing;!] 



|55^?^2^^^^?^/^ «eld test dassef. completed Unit II C!'! ^ 
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Unit 1 1 , Subtest 5,inergy 

Table 25 shows a summary of Sub- 
test's statistics followed by Individual 
items and their scoring Mys,; and the 
percentage of students 5n: the total 
group who selecteii each option on the 
pre* : and postteu* Table 26 shows the 
percentage of /students by / class who 
chose the ban resRpnse to each item. V 

Of the seven items related Ito 
energy-twoVtems C Items 18 and 19) 
assessed understanding that energy is 
required to do work. These two items 
proved deceptively easy in that most 
students could answer them prior to > 
instruction. Posttest ^levels of correct 
response were 91 and 85 . percent. 

(Continued) 
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EiMrgy (Continufd) 
^T^v Anc^lwf Item (Item 21)* assessed 
^ tife awareness that the higher the tern* 
perature oia Ii<juid/the more work it 
can do« Only 40 percent of the students 
k(^ew this <m the poittest.%; 

Two Items proved quite difficult 
; foe students. One of thwn (Item 28) 
j^sso^sed the recognition o? things 
; ^c^taining stored energy «AlnrK>st none 
of the students answered ihli item sue- 
i oessfutly. the other Uti?m 12) assessed 
; the knowledge that living things ulti> 
mateSy get their energy from the sun. 
Less ^an or>e fifth of the students were 
able to ansvi^rlth ii item correctly on the 
posttest^ representing no change from 
responses on the pretest. Neither of 
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these items assessed spsdfiv points of 
learning from direct instruction. 
Instead they required student^ to se^X 
the implications or make inferences 
about what had been presented «o dass. 
In the revision both of these concept! 
were judged to be peripheral to the 
m^or thrust of the unit^and noayempt 
was made to devise activities to teach 
thisinformatiorv 

The remaining two items in this 
subtest (Items 9 and 31) ass^^ssed (he 
knowledge that food '^ie body's 
source of energy. From half to two 
thirds of the students were able to 
answer these Items correctly on the 
posttest« representing considerable 
gains from pretest scores. 

Table 26 indicate^ the percentage 
of students ifti^each dass whose |otal 
uibtest score wa»^ore than one stan* 
dard deviation abcve or below the total 
group mi^an. This. table shows the per- 
centage of high performers (ansvwring 
more than five items correclfy^J and low 

9. ' 
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0<eff ormers (answering fewer than three 
items cw-rectly). Note thatit was possi- 
ble to earn partial credit- on Item 28. 
* Considering all of th^e Items as a 
subtest; 14 percent of ^e students 
answered more than five it^s correctly 
on the pofttest. while 20 percent 
answered fewer than three items cor* 
rectly (mean: 18.5, S-D.: 7, maximum 
score possible:: 35). 

In only one class "were one third or 
more of the students able to apswer 
more than five items correctly. How- 
ever, if one discounts the two quite dif* 
ficulTitems which ^re not a part of 
direct inVtruction, the average perfor- 
mance on\this subtest was at the rela- 
tively high level of more than three of 
the five remaining items answered 
correctly. Revisions of the materials 
included breaking activities into smaller 
steps, experience with more forms of 
energy, arKl some practical applications 
of the idea that thAitgher the tempera- 
ture, the more energy is present. The 
section of activities relating to food as 
the body's source of energy oAras 
expanded with additional activities on 
the energy values of va^ous foods. A 
game was developed called /'The Full 
and Healthy Game*' dealing with bal* 
anced diets and daily food and energy 
requirements* , * 



TABLE 26 

Subtest 5 (Energy): Percent Choosing the Correct Response, by Class 
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Unit I i, Subtest 6, Food Chains 

Several of the topic areas into 
which Unit ll items were divided con- 
tain only one or two items./fisljch cas^ 
the label ^'subtest" has still been 
applied; however/ tbe small number of 
items precludes nriuch anclysis as a sub- 
test. This subtest contains two items. 
See Table 27 for relevant statistics, 
items, and results. Table 28 shows per- 
formance by class and also reports suc- 
cess of individual students in respond- 
ing to both items. " / 

One item (Itehn 6) dealt with the 
ability to identify the appropriate links 
in a fc*d chain. Only 10 perce;n of the 
students were able to draw arrows to 
link the organisms approoc/ately in a 
simple food chain. Theother item (Item 
14) dealt with the recognition of the 
term **food web" as the appropriate 
term to describe the interrelationships 
(Continued) 
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Food Chains {Continued) 
of a number of living things. About half 
of the students correctly related the 
term to an illustration of a food web on 
the posttest. Considering both items 
a sul)t^t« only eight percent of thestu-. 
dents answ<»red more than one item 
correctly, while 42 percent answered 
neither item. In five of the ten classes, 
one third or more of the students were 
: unsuccessful in answering either item 
(mean: 3, S.D.: 3, maximum score 
possible: 10). 



activities ha)e been developed relating 
to food chains and webs and the interre- 
lationships of plants and arHmals in an 
ecosystem. This sequence of activities 
attempts to tie these relationships more 
closely to" the students' own lives. 



A number of additions and revi- 
sions of the maftertals have been made in 
order to establish the concepts more 
effectively. In v^at is now Unit ill* a 
new activity deals with what various 
animals eat. A food chavn game has been 
revised to provide experiences in linking 
animals to things that they eat and uitt- 
cmatcly to the sun. More experience 
relateb to food webs has been provided 
ir> several other activities. A new picture 
booklet deals with the interrelation* 
ships of plants ^d animals in an eco* 
system, although *'ecosystehi*' is.not 
uped in^the materials. In Unit IV, eight 
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Subtest 6 (Food Chains and Webs): Percent Choosing the Correct Response, ty Class 

t Test Classes 



Question 
» 


00 

i £ 


OS 

& & 

10 4 


06 

at - v» . 

t 

10 9 


L 

t & 

17 10 


•08 - 

n 9 


09^ ^ 

4-* 

12 n 


11 

M 

> > 9 


'-as 


11 






6 
14 


6 0 
13 73 


0 0 

0 sa* 


0 * 30 
20 90 


0 U 
29 73 


0 0 

25 60 i 


6 0 
17 9 


9 
18 


^ 0 


54 


V/* ' 





Level of , Posttest only Subtest Performance* 

Performance - 



High ; 
(more than one 
item correct} 


14 

•* ■ 


25 


33 


20 


0 




'o 


0 


0 


4 






Low 

(none correct) 


* 29 


i 


11 


30 


33 


82 


78 


* 47 




68 






Unit it. Subtest 7, Food Energy 

Five items were included in this 
subtest. Three of the items allowed par- 
tial credit for some responses. The two 
iterns which students found most diffi- 
cult required a written response. See 
Table 29 for a summary of subtest sta- 
tistics^ Items/ and responses* Table 30 
reports results by class. 

Three of the items (Items 2, 8,and 
24) assessed the knowledge that pl^dnts 
make their owrt Iool\ from sunlight and 
other material^. One of these ihree^i 
items (Item 2) rec^\red the students to ; 
state that plants make th&r own food or 
that they need air, water, soil, and sun* 
lighVto get their food. One fifth of the 
students stated this on the posttest, 
whereasi onby one percent clsn^ do so 
on tto pretest. On the other two of 
these three ^ems, about one half of the 
students marked a response that sun^ 
ii^ht was the plant's source of energy, 
and that plants could change the sun's 
: energy into food energy. ! * 

(O^ntinued) 
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Food Enerty (Continued) 

ii The remaining two items in this 
subtest (Items 1 and 15) assessed the 
understanding that plants are the uiti- 
mate source of all food for all living 
things^ and that without plants we 
would die. On!y six percent of the stu* 
dents expressed this answer in written 
form on Item 1; however, half of the 
^students were able to identify piants as 
the things that the animals depended 
upon in lten^15« an illustration of a 
foodweb. 
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: Table 30 reports the results by class 
■ for all five items combined. It {hows the 
percentage of students who4 perfor* 
manoe was htgh*(more than three items 
correct) or low (fewer than two items 
;',correct). ■ v^^■;■>^'...■.:■^^^ ■ 

Consi the five items as a sub* 
test, 10 percent of the students answer* 
ed more than three of the items correct* 
ly, while IB percent answered fewer 
than two items correctly (roiean: 11, 
S.D.: 6, maximum sdife possible: 25). 
One third of the students in tv^ of the 
classes were high performers on this 
Subtest, Overall^ the level of perfor- 
mance was reasonable considering the 
difficulty of the concepts. ^ 
A series of plant experiments were 
used to communicate many of the* ideas 
in this subtest; these experiments have 
been considerably revised. The 
sequence of the materials has also been 
changed, in revision. 



TABLE 30 ■ ; 
Subtest 7 (Food £tnergy): Percent Choosing the Correct Response, by Class 
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Unit II « Subtest 8, Weight and 
Temperature V 

Two Items were included in this subtest. 
One of the items (Item 23) assessed an 
understanding of how a t>alance should 
be used. The other item (Item 30)^ 
served to educate studenti' general 
understanding of what numbers in the 
Fahrenheit scale mean. As in all sub* 
tests, no attempt was made to fully 
assess student understanding. Instead 
tfiese two items merely sampled under- 
standings of measurement and.served as 
indicators rather than an inventory of 
what had been leamed. Table 31 sum* 
marizes subtest statistics^ and Table 32 
shows student performance on p^th 
items combined^bydas^. 

. in only three classes did more than 
two thirds of. the class succeed on Item 
23, understandirig the balance. On Item 
30, room temperature, less than half the 
total groupgave the correct response. In 
two classes,. however, more than two 
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thirds of the dass vyere successful. Item 
30 was drawn from sdence items used 
in the 1970 National Assessment of 
EducatbnaS Progress.' Results f6r the 
nine* and 13'year old national sample \ 
are shov/n on the item. As can be noted, 
i the response of the 13- to 14-yeaf-old 
^EMH field test group resembles the ; 
imponse of nine-year-cVi normal chilv 
dren on this item. 

Considerir>g 'the two items together 
as index to student understanding of 
measurement, about one third of the 
^ totall (group of students an:;wered both , 
items correctly, while one fifth answer- 
^ ed neither Item correctly. For p«rf or- 
manoe by dass, one third or more of the 
students iri^five dasses were high pe^ i 
formers, while about one third of the i 
students in two^lasses were unsuocess- I 
ful. )n only two' classes were students I 
predominantly successful on these | 
items with no studerits unsuccessful. 
^ *n the reused <naterials, to save th# 
time spent oorutructing balances, an i 
inexpensive commercial balance has 
been selected; rnore time is devoted to ; 
learning to read the balance and to 
measuring different things. ' ' ^ 
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■ * TABLE 32 

Subtest 8 (Weight and Temperature): Percentage Choosing the Correct Response, by Class 
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Unit i Subtest 9, Graphing 

: i White s>ifveral items had been coiv 
structed to assess a student's under* 
standing of graphing, defects in sonne of 
the itenns reduced the usable number of 
items to one. This item, drawn from 
science items used in the 1970 National , 
Assessrnent of Educational Progranns, 
assesfied the ability to recognize the cor* 
rect graphing of three Items of informal 
tion* Results are shown on the item for 
the total group lr\ this field test ancl for 
the nine-year-old <group sampled in the 
National Assessment study. Table 33 
shows percentage of students by dass 
choosing the best response to this item. 



Sixty seven percent of the group 
correctly answered the item on the 
posttest, representing little change from 
the pretest. This result for 13- to 14- 
year^ld children was dearly above the * 
performance of nine-year-old normal 
children in the National Assessment 
sample. It is probably not much below 
wb^t normal 13-Year-old children can 
do. For four field test dasses, £;bout 
four out of five students answered the 
item correctly; half the students in 
three other classes were unsuccessful. - 

These results reflected a modci- 
; ately high level of understanding. How- 
ever, items in Subit^ts 1 ar>d 3irxiicated 
student difficulty with graph interpre- 
tation^ Since graphs are used in various 
activities in the materials to summarize 
and compare data; it was considered 
necessary to develop more fully the 
concept of graphing. A new activity has 



been devised which provides practice in ■ 
graphing and reading graphs of various 
kinds. Other activities vyhich use graphs 
were expanded to provide more empha* 
SIS on the graphing process itself, 
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Subtest 9 (Graphing): 



Perccntt^e'ChDosIng the Correct Response, by Class 



Unit II, Subtest 10, Categorizing 

fTKis subtest consisted of two items 
which invotved not only the process of 
grouping things but an understanding of 
specific concepts. Qne item (Item 7) 
assessed whether students recognized 
seeds as living things. The other (Item 
35) required selection of a balanced 
meal from four pictured menus/ Table 
34 shows a sumimdry of subtest statis- 
tics. Table 35 shows the percentage of 
students by dass who chdse the best 
response to eadi item. . ' 
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' These two itefms were answered^ 
-correctly by approximately half of the 
field test group. No class stood out 
above the others in performance on 
ttiese items. About one tntrd of the stu- 
^ dents in the total group answered both 
items correctly* In interviev^ about 
Item 35, only one in four stodents (oit a 
random sample) gave a reason for mar'k* 
ing their answer the way they did and 
explained what a balamced meal was. 
The others either indicated the meal 
was one theyitked to eat, that it gave 
them energy, br that thf^re was m6re of 
it«andsoont - 

|n the revised n)ateria)s, a "Full and 
Healthy Game" was created to develop 
the concept of a balanced diet. An activ- 
ity was also added which requ/ire/ stu- 
dents to grow plants from seeds. 

In summary, performance as 
measured by the J9 items related to 
^Unit II of ME AND MY ENVIRON- 
^ MENT was moderate. In six of the 
classes one third or more of the^stude'nts 
were highly successful on at least one of 
the subtests. Considerable revision of 
Unit II mMerials should result in both a 
higher level of success and a greater 
number of students experiencing a jus- 
tifiable degree of success. 
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Percentage Choosing the Correct Response* by Class 
Field Test Classes 
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HAAG. Ida Patterson School, Eugene, Oregon; JC^RY HUBSCHMAN, l^right State 
University, Dayion, Ohio; CAN D ACE L}CHT,^ot^/l://ia Jr. High School, i^estminster, 
Colorado^ KEITH MORItlLL. South Dakota State University, Brookings* South 
: DaAroea; ALICE OGURA, 5aA»rar Environmental Center, Denver, Colorado; JOAN 
PETERS. Boulder, Colorado; NORRIS M. ROSS, JR^ fVicolet High School, 
Milwaukee, Wisconsin. '--..y ■ ■,' : r- ■■■ .■ 



1973 SUMMER WRITING TEAM 

JUNE E. ALLBRANDT, Lake Jr. High School, Denver, Colorado; LOUIS R. 
CLERICO, Hillside School, Montclair, New Jersey; STELLA GUBBINS. Boulder, 
Colorado; FLORENCE B. HAAG, Ida Patterson School, Eugene, Oreyon;WARREN E. 
HEISS, Afonrc/a/r State College, Upper Montclair, New Jersey; JERRY H. 
HUBSCHMAN, IVr/p/if State University, Dayton, Ohio; CANDACE J. LIGHT, 
Hodgkins Jr. High School,^Westnynster, Colorado; KEITH MORRILL, 5o(/r/> Dakota^ 
State University, Brookings, South Dakota; XjEORGE E. NOWAK, Madison High 
School, San Diego, California; MARY D, SMITH, Shepard Jr, High School, Durham, • 
North Carolina. ' :■ ■ ■ , 
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FIELD TEST TEACHERS * 

JUNE E. ALLBRANOT. L§k9Jr, High Schoot. Denver, Colorado; LARRY E. ALLEN, 
Sentinef High School, Missoula. Monjana: VINCENT J. ALVINO. Gofeta Valley Jr. 
High SchooK Coleta. California: ALICE 6. HXQHM^^AycackJr. High School, Baleigh, 
North Carolina: DANNETTE BOYLE.TVtfwVi Piatt Jr. High School. Boulder, Colorado: 
CHAMINAOE C FARMER^ Gibson Jr. High School, Us Vegas, Nevada: SHtLOOH X 
FINE. C. -t. S, 148, Bronx, New York; MARY L. PAEEMAN, Hodgkins Jrl High 
Schoof, Westminster, Colorado: CARL E. HOFF^ Lafayette Middle School, Lafayette. 
"Colondb; ALMA W. JENKINS, Skiles Middle School, Bwartsfoo, Illinois; JO ANN 
JEPPSON, Granite Part Jr. High School, Salt Lake City, Utah; STEVE G. JOHNSON, 
Central School, Missoula^ Montana; WALLtS M. KIRYLUK, Cote Jr. High School, 
Denver, Coldrado; MOLLIE P. KITE, Pathfinder School, BetheLBark, Pennsylvania; 
CANCACE J. LIGHT. Hodgkins Jr: High School, kVestminster, Colorado; CECIL C. 
LINOER. f^ooseve/i Jr. High School,' Bugene, Oregon; VITO LOMBAROO. 0. R. 
Middle School. Bnttol, /'irnnsK/Mn/j; ' R ICHARO VATHIS, Shadowlavvn Learning 
Center, Arlington, Tennpsee; EDWARD H. McCANN, Mon Valley School, West 
Mifflin, Pennsylvania: MOLLY McCARTHY, Beryamin franklin Jr. High School, San 



Francisco, California; R.^ DOUGLAS MoCULLOUGH, ^, Y. Cairns Vocation^ School, 
Bdmonton, Alberta. (Canada; EDWAR3 F, McNULTY, Eastwood Jr^ High School, 
Syracuse, New York; JOAN AJ McNULTY, Central Jr. High School, OeWitt, Iowa; 
MICHAEL W. RAt, Granite Park Jr. High School, Salt Lake City, Utaji' EVA G. 
REEDY. Benfanun Franklin Jr. High SchooK San Francisco, California: TOM E. 
RODGERS, GarsideJr. High School, Las Vegas, Nevada; tWif^M H, SHERMAN,/. 
S. 131, Bromc, New York; HOWARD E. SHIPLEYy/orfjo/i Hill School, Morristown. 
Tennessee; EDITH D, ^HIPMAN, Pacific Prevocfftonal School, Seattle, Washington; 
B^rrV S/SILVERTHORN, Selah Jr. High Sc^p/, Selah, Washington; MARY 0.' 
SMITH, Shepard Jr. High School, Durham, Nortfi Carolina." JOSEPH SOUSA, 
Harrington Way Jr. High School, Worcester, Massachusetts; FRED W. STRICKLAND, ' 
Nathan Weeks Jr, High School, Des Moines. Iowa; HILDA S. THACH. Pierson Jr, High 
School, Kansas City; Kansas; SUSAN K« THOMASGARO. Merrill Jr. High School, 
i Oinver. Colorado; CARLA J. WATTS, VNC Lab School, Greeley, Colorado; MARY S. 
WHITE, Shawnee Jr. High School, Louisville, Kentucky; SUE WRIGHT^ Wnt Locust 
elementary School, Wilmington,. Ohio; AUGUST ZEITLOW, UNC lab School, 
G reef ey, Colorado. 



